Modeling Valence Effects in Unsupervised Grammar Induction

About this report: While the results of this paper are not entirely positive, I feel it contains some valuable ideas. I n hopes that others will find these helpful or interesting, I'v e published this as a technical report. It is still my belief th at the poor multilingual performance is due to a bug in the implementation, but I do not currently have time to investigate this. This work was done during the 2006-2007 school year (partially at Charles University in Prague). Abstract We extend the dependency grammar induc- tion model of Klein and Manning (2004) to incorporate further valence information. Our extensions achieve significant improve- ments in the task of unsupervised depen- dency grammar induction. We use an ex- panded grammar which tracks higher orders of valence and allows each valence slot to be filled by a separate distribution rather than using one distribution for all slots. Addition- ally, we show that our performance improves if our grammar restricts the maximum num- ber of attachments in each direction, forc- ing our system to focus on the common case. Taken together, these techniques constitute a 23.4% error reduction in dependency gram- mar induction over the model by Klein and Manning (2004) on English.