LDC data and commercial technology development
New Publications:
BOLT Egyptian Arabic-English Word Alignment -- Conversational Telephone Speech Training
BOLT Egyptian Arabic-English Word Alignment -- Conversational Telephone Speech Training
__________________________________________________________________
Spring 2020 LDC Data Scholarship recipients
LDC congratulates the following Spring 2020 Data Scholarship recipients:
Spring 2020 LDC Data Scholarship recipients
LDC congratulates the following Spring 2020 Data Scholarship recipients:
- Zahra Azin (Istanbul Technical University, Turkey) is awarded a copy of Abstract Meaning Representation (AMR) Annotation Release 3.0 (LDC2020T02) for her work in Turkish AMR.
- Spandan Dey (IIT Kharagpur, India) is awarded a copy of Multi-Language Conversational Telephone Speech – South Asian (LDC2017S14) for his research on automatic language recognition.
- Jonathan Downey (University of California, Santa Barbara, US) is awarded a copy of the ETS Corpus of Non-Native Written English (LDC2014T06) for his research on second language acquisition and quantitative methodologies for educational measurements.
- Nathaniel Fackler (University of Georgia, US) is awarded a copy of the ETS Corpus of Non-Native Written English (LDC2014T06) for his work on adult second language acquisition.
- B. Senthil Kumar (SSN College of Engineering & Anna University, India) is awarded a copy of 2009 CoNLL Shared Task Part 2 (LDC2012T04) for his research on semantic role labeling.
- Ming Li (Colorado School of Mines, US) is awarded a copy of TIDIGITS (LDC93S10) for her research on inferring speech signals from motion data in Internet of Things (IoT) security.
- Jialiang Lin (Xiamen University, China) is awarded a copy of the ETS Corpus of Non-Native Written English (LDC2014T06) for his project to train and test an automated essay scoring model.
Students
can learn more about the LDC Data Scholarship program and the next application
cycle on the Data Scholarships page.
LDC data and commercial technology development
LDC data and commercial technology development
For-profit organizations are reminded that an LDC membership is a
pre-requisite for obtaining a commercial license to almost all LDC databases.
Non-member organizations, including non-member for-profit organizations, cannot
use LDC data to develop or test products for commercialization, nor can they use
LDC data in any commercial product or for any commercial purpose. LDC data
users should consult corpus-specific license agreements for limitations on the
use of certain corpora. Visit the Licensing page for
further information.
__________________________________________________________________
New publications:
The data in this release was collected by LDC at its Human Subjects Data Collection Laboratories in Philadelphia and by the International Computer Science Institute (ICSI) at the University of California, Berkeley, as a collaborative, carefully coordinated activity at both recording sites. The Mixer 4 and 5 collection contains 2,568 recordings made via the public telephone network and 2,152 sessions of multiple microphone recordings in office-room settings.
The telephone protocol connected recruited speakers through a robot operator to carry on casual conversations. In Mixer 4, 400 subjects made ten 10-minute calls; half of those subjects also visited one of the collection sites where they made two telephone calls while also being recorded on a cross-channel platform. In Mixer 5, 300 subjects each completed ten calls and six interview sessions at either LDC or ICSI; those sessions were conducted on a cross channel platform and included a telephone call in one of three vocal-effort conditions - normal, high and low. Mixer participants were nearly all native English speakers, the rest being bilingual English speakers.
This release includes metadata about the calls and speakers, along with time-aligned entries for many of the component portions of the recording sessions.
Mixer 4 and 5 Speech is distributed via hard drive.
2020 Subscription Members will automatically receive copies of this corpus. 2020 Standard Members may request a copy as part of their 16 free membership corpora. This corpus is a members-only release and is not available for non-member licensing. Contact ldc@ldc.upenn.edu for information about membership.
New publications:
(1) BOLT Egyptian Arabic-English
Word Alignment -- Conversational Telephone Speech Training was developed by
LDC and consists of 153,171 words of Egyptian Arabic and English parallel text
enhanced with linguistic tags to indicate word relations.
The source data in this release consists of transcripts of Egyptian Arabic conversational telephone speech (CTS) from LDC's CALLHOME and CALLFRIEND collections (LDC97S45, LDC97T19, LDC2002S37, LDC2002T38, LDC96S49) that was translated into English by professional translation agencies and annotated for the word alignment task.
The BOLT word alignment task was built on treebank annotation. Egyptian Arabic source tree tokens were automatically extracted from tree files in LDC’s BOLT Egyptian Arabic Treebank, which had been tagged for part-of-speech and syntactically annotated. That data was then aligned and annotated for the word alignment task.
BOLT Egyptian Arabic-English Word Alignment -- Conversational Telephone Speech Training is distributed via web download.
2020 Subscription Members will automatically receive copies of this corpus. 2020 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for a fee.
The source data in this release consists of transcripts of Egyptian Arabic conversational telephone speech (CTS) from LDC's CALLHOME and CALLFRIEND collections (LDC97S45, LDC97T19, LDC2002S37, LDC2002T38, LDC96S49) that was translated into English by professional translation agencies and annotated for the word alignment task.
The BOLT word alignment task was built on treebank annotation. Egyptian Arabic source tree tokens were automatically extracted from tree files in LDC’s BOLT Egyptian Arabic Treebank, which had been tagged for part-of-speech and syntactically annotated. That data was then aligned and annotated for the word alignment task.
BOLT Egyptian Arabic-English Word Alignment -- Conversational Telephone Speech Training is distributed via web download.
2020 Subscription Members will automatically receive copies of this corpus. 2020 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for a fee.
*
(2) EVALution was developed
by The Hong Kong Polytechnic University. It is comprised of English and Mandarin
Chinese data sets -- EVALution 1.0 and EVALution-Man, respectively -- that
contain semantic relations and metadata for training and evaluating
distributional semantic models.
EVALution 1.0 consists of approximately 7500 English tuples extracted from ConceptNet 5.0 and WordNet 4.0 and filtered through automatic methods and crowd-sourcing. Several semantic relations between word pairs were instantiated, including hypernymy, synonymy, antonymy and meronymy. The corpus also includes additional information that can be used to filter the pairs or to analyze the results, such as relation domain, word frequency, word part-of-speech and word semantic field.
EVALution-MAN consists of Chinese word pairs from two sources: Chinese Wordnet and humans who completed an elicitation task by supplying missing words to sentences. The human-supplied sentence word pairs were then judged by human raters for reliability.
EVALution is distributed via web download.
2020 Subscription Members will receive copies of this corpus provided they have submitted a completed copy of the special license agreement. 2020 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for a fee.
(3) Mixer
4 and 5 Speech was developed by LDC and contains approximately 14,185 hours
of audio recordings of conversational telephone speech, interviews, elicitation
exercises and transcript readings involving 616 distinct speakers. The material
was collected in 2007 as part of the Mixer project – which supported speaker
recognition for a variety of research tasks – and recordings in this corpus
were used in the 2008 NIST Speaker Recognition Evaluation.
EVALution 1.0 consists of approximately 7500 English tuples extracted from ConceptNet 5.0 and WordNet 4.0 and filtered through automatic methods and crowd-sourcing. Several semantic relations between word pairs were instantiated, including hypernymy, synonymy, antonymy and meronymy. The corpus also includes additional information that can be used to filter the pairs or to analyze the results, such as relation domain, word frequency, word part-of-speech and word semantic field.
EVALution-MAN consists of Chinese word pairs from two sources: Chinese Wordnet and humans who completed an elicitation task by supplying missing words to sentences. The human-supplied sentence word pairs were then judged by human raters for reliability.
EVALution is distributed via web download.
2020 Subscription Members will receive copies of this corpus provided they have submitted a completed copy of the special license agreement. 2020 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for a fee.
*
The data in this release was collected by LDC at its Human Subjects Data Collection Laboratories in Philadelphia and by the International Computer Science Institute (ICSI) at the University of California, Berkeley, as a collaborative, carefully coordinated activity at both recording sites. The Mixer 4 and 5 collection contains 2,568 recordings made via the public telephone network and 2,152 sessions of multiple microphone recordings in office-room settings.
The telephone protocol connected recruited speakers through a robot operator to carry on casual conversations. In Mixer 4, 400 subjects made ten 10-minute calls; half of those subjects also visited one of the collection sites where they made two telephone calls while also being recorded on a cross-channel platform. In Mixer 5, 300 subjects each completed ten calls and six interview sessions at either LDC or ICSI; those sessions were conducted on a cross channel platform and included a telephone call in one of three vocal-effort conditions - normal, high and low. Mixer participants were nearly all native English speakers, the rest being bilingual English speakers.
This release includes metadata about the calls and speakers, along with time-aligned entries for many of the component portions of the recording sessions.
Mixer 4 and 5 Speech is distributed via hard drive.
2020 Subscription Members will automatically receive copies of this corpus. 2020 Standard Members may request a copy as part of their 16 free membership corpora. This corpus is a members-only release and is not available for non-member licensing. Contact ldc@ldc.upenn.edu for information about membership.
*
