LDC September 2026 Newsletter

LDC data and commercial technology development        

New publications:

CALLHOME Mandarin Chinese Second Edition

CALLHOME Mandarin Chinese Lexicon Second Edition

MATERIAL Lithuanian-English Language Pack

_____________________________________________________________________

LDC data and commercial technology development
For-profit organizations are reminded that an LDC membership is a pre-requisite for obtaining a commercial license to almost all LDC databases. Non-member organizations, including non-member for-profit organizations, cannot use LDC data to develop or test products for commercialization, nor can they use LDC data in any commercial product or for any commercial purpose. LDC data users should consult corpus-specific license agreements for limitations on the use of certain corpora. Visit the Licensing page for further information.

New publications:

CALLHOME Mandarin Chinese Second Edition was developed by LDC and contains 38 hours of speech from 120 unscripted telephone conversations between native Mandarin Chinese speakers. This publication is a re-release of the original CALLHOME Mandarin Chinese collection, combining CALLHOME Mandarin Chinese Speech (LDC96S34) and CALLHOME Mandarin Chinese Transcripts (LDC96T16), with additional transcription and updated directory structure, file formats, and documentation.

This release contains the 120 telephone conversations published in CALLHOME Mandarin Chinese Speech (LDC96S34) which represented training and development data and a subset of evaluation data. Participants spoke on topics of their choice in a single telephone call lasting up to 30 minutes. Calls were manually audited for gender, language, recording quality, channel characteristics, dialect, and accent. For this second edition, all audio was converted from SPHERE files to FLAC format, and the original training/development/evaluation partitioning was removed. 

This release also features revised transcripts conforming to updated LDC transcription guidelines that addressed normalization of annotation formats, standardization of speaker-produced and background noises, application of foreign-language marking, whitespace cleanup, and corrections and consistency fixes.

The CALLHOME series consists of telephone conversations and transcripts developed by LDC and Rutgers, The State University of New Jersey, in support of research in speaker identification, language identification and related technologies. Languages in the series include American English, Egyptian Arabic, German, Japanese, Mandarin Chinese, and Spanish.
2026 members can access this corpus through their LDC accounts. Non-members may license this data for a fee. 

*

CALLHOME Mandarin Chinese Lexicon Second Edition was developed by LDC and contains 44,404 Mandarin Chinese words with morphological, phonological and frequency information. This second edition updates file formats, directory structure and documentation. The first edition is available as CALLHOME Mandarin Chinese Lexicon (LDC96L15). 

The words in the lexicon were derived from transcripts representing unscripted telephone conversations between native Mandarin Chinese speakers contained in CALLHOME Mandarin Chinese Second Edition (LDC2026S11) and from Chinese news text.

The lexicon contains seven tab-separated information fields: (1) headword: orthographic representation of the word in hanzi (e.g., 没有); (2) pinyin: headword transcribed in pinyin (e.g., mei2 you3); (3) tone: tone sequence for headword (e.g., 2 3); (4) pron: pronunciation of headword without tone information (e.g., mey yow); (5) pos: part-of-speech tag for the headword; (6) xinhua_freq: frequency of the headword in newswire text; and (7) train_freq: frequency of the headword in the CALLHOME transcripts. It is presented as a tab-delimited TSV file encoded in UTF-8 format and includes a pronunciation dictionary derived from the lexicon in UTF-8 encoded CMUdict format. 

2026 members can access this corpus through their LDC accounts provided they have submitted a completed copy of the special license agreement. Non-members may license this data for a fee.

*

MATERIAL Lithuanian-English Language Pack was developed by Appen for the IARPA MATERIAL program and contains 64 hours of Lithuanian conversational telephone speech, transcripts, English translations, annotations and queries. Calls were made using different telephones (e.g., mobile, landline) from a variety of environments. Transcripts cover approximately 100% of the speech files, 6% of which were translated into English. This release also includes domain annotations, English queries and their relevance annotations. 

The MATERIAL program focused on underserved languages with the ultimate goal to build cross language information retrieval systems to find speech and text content using English search queries.

2026 members can access this corpus through their LDC accounts provided they have submitted a completed copy of the special license agreement. Non-members may license this data for a fee.

LDC 2018 December Newsletter

LDC Membership Discounts for MY2019 Still Available

Spring 2019 LDC Data Scholarship Program - deadline approaching

New publications:

HUB5 Mandarin Telephone Speech and Transcripts Second Edition
Nautilus Speaker CharacterizationTAC Relation Extraction Dataset
_______________________________________________________________

LDC Membership Discounts for MY2019 Still Available

Join LDC while membership savings are still available. Now through March 1, 2019, renewing MY2018 members will receive a 10% discount off the membership fee. New or non-consecutive member organizations will receive a 5% discount. Membership remains the most economical way to access LDC releases. Visit Join LDC for details on membership options and benefits.

Spring 2019 LDC Data Scholarship Program - deadline approaching 

Students can apply for the Spring 2019 Data Scholarship Program now through January 15, 2019. The LDC Data Scholarship program provides students with access to LDC data at no cost. For more information on application requirements and program rules, please visit LDC Data Scholarships. 

New publications:

(1) HUB5 Mandarin Telephone Speech and Transcripts Second Edition was developed by LDC in support of US government projects for language recognition and Large Vocabulary Conversational Speech Recognition (LVCSR). The first edition was released by LDC in two data sets, HUB5 Mandarin Telephone Speech Corpus (LDC98S69) and HUB5 Mandarin Transcripts (LDC98T26). This second edition merges the speech and transcript releases, updates the audio format, and adds Pinyin transcripts, forced alignment, and updated documentation and metadata.

This corpus contains approximately 19 hours of Mandarin speech from 42 unscripted telephone conversations between native speakers of Mandarin from CALLFRIEND Mandarin Chinese-Mainland Dialect (LDC96S55), which has also been released in a second, updated edition (LDC2018S09) and (2) associated transcripts of contiguous 5-30 minute segments from those telephone conversations.

Participants could speak with a person of their choice on any topic; most called family members and friends. The recorded conversations lasted up to 30 minutes. Transcripts were created manually by native Mandarin speakers in the GB2312 encoding schema. This release includes Pinyin transcripts and the original transcripts, both in UTF-8 format. 

HUB5 Mandarin Telephone Speech and Transcripts Second Edition is available via web download. 

2018 Subscription Members will automatically receive copies of this corpus. 2018 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for a fee.

*

(2) Nautilus Speaker Characterization was developed at the Technical University of Berlin and is comprised of approximately 155 hours of conversational speech from 300 German speakers aged 18 to 35 years (126 males and 174 females) with no marked dialect or accent, recorded in an acoustically-isolated room. The corpus was designed to support research on the detection of speaker social characteristics, such as personality, charisma, and voice attractiveness.

Four scripted and four semi-spontaneous dialogs simulating telephone call inquiries were elicited from the speakers. Additionally, spontaneous neutral and emotional speech utterances (predominantly excitement or frustration) and questions were produced.

Speech corresponding to one of the semi-spontaneous dialogs was evaluated with respect to 34 continuous numeric labels of perceived interpersonal speaker characteristics (such as likable, attractive, competent, childish). For a set of 20 selected "extreme" speakers evaluated for their warmth-attractiveness, 34 naive voice descriptions (such as bright, creaky, articulate, melodious) were also evaluated. The corpus contains all labels, together with the speech recordings and the speakers' metadata (e.g., age, gender, place of birth, chronological places of residence and duration of stay, parents' place of birth, self-assessed personality).

Nautilus Speaker Characterization is available via web download. 

2018 Subscription Members will receive copies of this corpus provided they have submitted a completed copy of the special license agreement. 2018 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data at no cost.

*

(3) TAC Relation Extraction Dataset (TACRED) was developed by The Stanford NLP Group and is a large-scale relation extraction dataset with 106,264 examples built over English newswire and web text used in the NIST TAC KBP English slot filling evaluations during the period 2009-2014. The annotations were derived from TAC KBP relation types (see the guidelines), from human annotations developed by LDC and from crowdsourcing using Mechanical Turk.

Source corpora used for this dataset were TAC KBP Comprehensive English Source Corpora 2009-2014 (LDC2018T03) and TAC KBP English Regular Slot Filling - Comprehensive Training and Evaluation Data 2009-2014 (LDC2018T22). For detailed information about the dataset and benchmark results, please refer to the TACRED paper.

TAC Relation Extraction Dataset is available via web download. 

2018 Subscription Members will automatically receive copies of this corpus. 2018 Standard Members may request a copy as part of their 16 free membership corpora. Non-members may license this data for a fee.