New publications:
LDC2012T02
- English Translation Treebank: An Nahar Newswire -
- English Translation Treebank: An Nahar Newswire -
LDC2012S04
- Malto Speech and Transcripts -
- Malto Speech and Transcripts -
Thanks to all who participated in the 2012 LDC Survey. Your responses were thoughtful and informative. We’re now analyzing the results; stay tuned for an announcement on the survey findings.
In the meantime, please join us in congratulating Todor Ganchev from the University of Patras, Wire Communications Laboratory (WCL) for winning the survey participation benefit! As a reminder, one $500 benefit was awarded to a blindly-selected participant whose response was received by February 7, 2012.
LDC will be traveling across the globe to exhibit at its first IEEE-hosted event. The 37th International Conference on Acoustics, Speech, and Signal Processing (ICASSP) will be held at the Kyoto International Conference Center in Kyoto, Japan, on March 25 - 30, 2012.
The ICASSP meeting is the world’s largest and most comprehensive technical conference focused on signal processing and its applications, and LDC is looking forward to interacting with members of this community. Please look for LDC’s exhibition at Booth #14 in the Annex Hall. We hope to see you there!
New Publications
(1) English Translation Treebank: An Nahar Newswire was developed by LDC and consists of 599 distinct newswire stories from the Lebanese publication An Nahar translated from Arabic to English and annotated for part-of-speech and syntactic structure.
This corpus is part of an ongoing effort at LDC to produce parallel Arabic and English treebanks. The guidelines followed for both part-of-speech and syntactic annotation are Penn Treebank II style, with changes in the tokenization of hyphenated words, part-of-speech and tree changes necessitated by those tokenization changes and revisions to the syntactic annotation to comply with the updated annotation guidelines (including the "Treebank-PropBank merge" or "Treebank IIa" and "treebank c" changes). The original Penn Treebank II guidelines, addenda describing changes to the guidelines and the tokenization specifications can be found on LDC's website.
The data consists of 461,489 tokens in 599 individual files. The news stories in this release were published in An Nahar in 2002.
The English sources files (translated from the Arabic) were automatically tokenized, part-of-speech tagged and parsed; the tokens, tags and parses were manually corrected. The quality control process consisted of a series of specific searches for over 100 types of potential inconsistency and parse or annotation error. Any errors found in those searches were manually corrected.
Annotations are in the following two formats:
- Penn Style Trees
- Bracketed tree files following the basic form (NODE (TAG token)). Each sentence is surrounded by a pair of empty parentheses.
- AG xml
- TreeEditor .xml stand-off annotation files. These files contain the POS and Treebank annotation and reference the source files by character offset. DTD files for the AG xml files were moved from their original location indicated in the readme to be more consistent with LDC publications.
English Translation Treebank: An Nahar Newswire is distributed via web download. 2012 Subscription Members will automatically receive two copies of this corpus on disc. 2012 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for US$4500.
*
(2) Malto Speech and Transcripts was developed by Masato Kobayashi, Associate Professor in Linguistics at the University of Tokyo (Japan), and Bablu Tirkey, research scholar at the Tribal and Regional Languages Department, Ranchi University (India). It contains approximately 8 hours of Malto speech data collected between 2005 and 2009 from 27 speakers (22 males, 5 females). Also included are accompanying transcripts, English translations and glosses for 6 hours of the collection. Speakers were asked to talk about themselves, their lives, rituals and folklore; elicitation interviews were then conducted. The goal of the work was to present the current state and dialectal variation of Malto.
Malto is a Dravidian language spoken in northeastern India (principally the states of Bihar, Jharkhand and West Bengal) and Bangladesh by people called the Pahariyas. Indian census data places the number of Malto speakers in a range of between 100,000-200,000 total speakers. Most Malto speakers live in the three northeastern districts of Jharkhand, i.e, Sahebganj, Godda and Pakur; the fieldwork that resulted in this corpus was conducted in those districts. Of the Pahariyas in that area, three subtribes, the Sawriya Pahariyas, the Mal Pahariyas and the Kumarbhag Pahariyas, primarily speak Malto.
The transcribed data accounts for 6 hours of the collection and contains 21 speakers (17 male, 4 female). The untranscribed data accounts for 2 hours of the collection and contains 10 speakers (9 male, 1 female). Four of the male speakers are present in both groups.
All audio is presented in .wav format. Each audio file name includes a subject number, village name, speaker name and the topic discussed. The transcripts and glossary are UTF-8 text files. Because of ambiguities that occur when writing Malto in Devenagari script, the transcripts were developed using Roman script with symbols adapted from the International Phonetic Alphabet (IPA) but are not considered phonetic transcripts.
Malto Speech and Transcripts is distributed on 1 DVD. 2012 Subscription Members will automatically receive two copies of this corpus. 2012 Standard Members may request a copy as part of their 16 free membership corpora. The first 100 copies distributed to non-member organizations are available at no charge. Shipping and handling fees apply.
