Using Third Party Applications to create Ingest Transcripts

Transcripts can be produced in any application that can export delimited text files that can be manipulated to describe:

• A legible timecode
• One or more speakers (optional)
• Plain text

Software for Producing Transcripts

Note that these instructions may not always be entirely up to date, and edits are welcome. This reflects our experiences of using the following software to produce the XML transcript ingest format used by the Oral History module. In each case, the tool is used, and then a delimited file is exported and manipulated in OpenRefine to create the final ingest format.

Inqscribe

This software can be purchased with an academic or student discount. https://www.inqscribe.com/buy.html. Jump to Inqscribe Overview

Inqscribe Overview

A good transcript created in Inqscribe

[00:00:00.00] Daku Sherpa: The transcription text goes here on the
same line. Never start a new line or a new paragraph. Always have a
closing time stamp.[00:06:32.12]
[00:06:32.12] Interviewer: More transcription goes here. [00:09:10.45]
[00:09:10.45] No speaker: (Pause) Daku sips his drink. [00:20:11.15]
[00:20:11.15] Interviewer: So did you ever end up going back there? [00:25:10.45]
[00:27:11.15] Unknown: It's too cold there. [00:30:10.45]

The Inqscribe Window

1. The video play window
2. The buttons and tools used to start, stop and change the time in videos. You can choose a specific time through the timescale ruler. You are also able to use the slider to change the play rate making the video slower for transcription.
3. The window where you type the physical transcription. Properly formatted timestamps are blue. If the timestamp is not blue that shows there is an error in formatting.

Steps for Creating a Transcript in Inqscribe

1. Open the Inqscribe program.
2. Drag and drop the video file that you want to transcribe into the grey box located on the top left of the program that reads “Drag a media file into this window”. For video use only .mov or .mp4 files.
3. Before you begin to transcribe the video you must insert a timestamp. To do this go to Edit > Insert Time (it will insert a timestamp in the format of [hh:mm:ss.ss]). You can also use the hotkey CMD + ; to insert the timestamp.
4. Type your transcribed text
5. Insert a timestamp at the end of the line of transcribed text by going to Edit > Insert Time
6. Repeat on new line to continue transcription.
7. Optional: When recording the audio of multiple speakers you can create create snippets that insert the timestamp and speaker's name. This will also be useful to for inserted repeating block of text. To do this go to Edit > Insert Snippet and a box titled “snippets” will appear. To add a new snippet to the list click the “edit…” button. To Insert a snippet into the transcript click the “insert…” button. The image of the snippet editor (figure x) can be seen below. Snippets are entered with the formula: {$TIME} The text you want in the transcript. An example of a snippet that identifies the speaker is: {$TIME} Speaker 1

Inqscribe transcription tips:

• Changing the time in the timestamp is possible. As long as you maintain the correct format (Inqscribe highlights a valid timestamp in blue).
• You can adjust play rate if you need to listen to a speaker carefully

Steps for exporting your Inqscribe transcript as a tab delimited file

Export your completed transcript as a tab delimited file

1. Save you work by going to file > Save as. Save the interview transcript with the same filename as the Interview Video format and put it in the appropriate directory and file folder.
2. Go to File > Export > Tab-delimited text. The Export Settings box will open (see figure below)
3. Ensure that the “Export Out Points” and the “Export Speaker Names” boxes are checked. In addition ensure that the Speaker Name Delimiter box has a : )See below for example).

4. Input the name of your file the Target box. Use the choose… button to select the appropriate file folder directory. Then click Export.

Creating Snippets for Shortcuts to Create Transcripts:

In the transcript example below there are multiple speakers. The snippet feature helps the transciber keep track of all the possible speaker names.

Audacity

Audacity is an open audio editing and recording program. Transcribing is possible in this program. You can transcribe the audio files as well as the audio from a video file (MOV/ MP4), but you will not be able to view the video in the process. This is a drawback if you believe there are important actions that need to be expressed in the text. Audacity is mainly a sound editing software, but it is possible to create a transcript in this program using the label feature. You can transcribe the audio from a video MOV or MP4 file but, you will not be able to watch the video. Audacity is a free program and can be downloaded from http://www.audacityteam.org/download/. [Jump to Audacity Overview]

Audacity Overview

The Audacity Window

1. The playback controls to start and stop the audio.
2. The magnifying glass to zoom in and out of the track. This is a very important feature because you will need to zoom in to fit in all of your labels.
3. The playback speed control slider. Use this tool to speed up or slow down audio during transcription.
4. The Track Pane. This pane will show you where you are in the interview. Click and drag to highlight a portion of audio you want to listen to or label.
5. The Label Pane. This is where you enter the transcription text as labels.

Steps for Generating the Transcript In Audacity

1. Open Audacity on your computer.
2. Open the file you are transcribing by going to File> Open and select the file from your directory. The track will open in the program.
3. In Audacity you can select a section of the track to annotate. First zoom in using the magnifying glass (This is an important step as it will allow your labels to fit on the track). Then select part of the tract by clicking the start of where you want to begin your annotation and drag the cursor to the end (see example below).
4. Listen to the selected audio by using the playback controls. If necessary you can slow down the audio using the slider.
5. Add the label by going to Tracts> Add Label at Selection (or use the hotkey ctrl + b (Windows)/ cmd + b (mac)). Note that the program automatically generates the time stamps.
6. Input the text into the label. Remember to record the speaker's name using the proper format of Speaker's Name: What they are saying. Please see below for an example.
7. When you have finished transcribing the audio you can read the transcript as a whole by going to Tracts> Edit Labels… Here you can correct text. Note you will not be able to listed to the audio while this window is open. It is good practice to review the labels while listening to the audio once the transcript is complete to ensure it is error free.
8. When you are finished Make sure you save your work by going to File > Save Project As…
9. Export the transcript by going to File> Export Labels. Save the file with same name as the original audio/video, but with the extension .txt.

YouTube

YouTube allows transcripts to be created for uploaded videos. The transcripts can be downloaded as .srt, .vtt and sbv files. As YouTube does not export transcripts as delimited files there is an added intermediate step of converting the file in OpenRefine. These can then be processed in openrefine to produce XML files.

oTranscribe (BETA)

oTranscribe allows for the transcribing of recorded interviews. There is a web based tool and a downloadable tool. The downloadable tool takes some setup.

Sample text-based output

0:01 old and in the way

0:24 they'll never care about you

0:31 once I hear tell he was happy

0:38 good time

0:44 looking back to a better day

Sample oTranscribe (json) format

{  
   "text":" <p data-l10n-id=\"\"><span class=\"timestamp\" data-timestamp=\"0:01\" contenteditable=\"false\">0:01</span> old and in the way<br></p><p data-l10n-id=\"\"><span class=\"timestamp\" data-timestamp=\"0:24\" contenteditable=\"false\">0:24</span> they'll never care about you <br></p><p data-l10n-id=\"\"><span class=\"timestamp\" data-timestamp=\"0:31\" contenteditable=\"false\">0:31</span> once I hear tell he was happy<br></p><p data-l10n-id=\"\"><span class=\"timestamp\" data-timestamp=\"0:38\" contenteditable=\"false\">0:38</span> good time<br></p><p data-l10n-id=\"\"><span class=\"timestamp\" data-timestamp=\"0:44\" contenteditable=\"false\">0:44</span> looking back to a better day<br></p>\n <p><br></p><p>\n </p>",
   "media-source":""
}

Transforming to the target ingest format

What you need for the Ingest Final Package

In this Section:

• Create MODS metadata

• Ingest into Fedora

Create MODS metadata

Create a table containing the metadata using google sheets

Follow the steps below to create MODS metadata pertaining to the video file to be ingested into Islandora:

1. Go to the Google Sheet:

   https://docs.google.com/spreadsheets/d/16AyEKRdqaWWj9Yeo9prmvZiKNonDEJg9FI2hLF26ukE/edit?usp=sharing.

2. Make sure to create a copy of the spreadsheet to work on for each

   metadata record created. To create a copy go to file>create a copy.

3. Enter the appropriate metadata into the copy starting at line 2

   overwriting the cells containing descriptions. Delete all metadata in the following rows that do not pertain to your video or audio recording.

4. Save your file as a tab delimited format by going to File>

   Download as > Tab-separated Variables (TSV)

Convert Metadata table in MODS XML

1. Open OpenRefine

2. Click browse and select the metadata TSV file you are working with.

   Click next> Create Project.

3. To begin the process of exporting the table to XML click

   Export> Templating. The Templating Export box will appear.

4. Copy and paste the text below into your the Templating Export box.

   Ensure that you match the appropropriate text to the input box using the bold headers. Then click the Export button. Give file the appropriate name and place in the appropriate directory.

Prefix

<mods xmlns="http://www.loc.gov/mods/v3" 
xmlns:mods="http://www.loc.gov/mods/v3"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xlink="http://www.w3.org/1999/xlink">

Row Template (IMPORTANT: replace sound_recording with videorecording if your oral history is a video)

<titleInfo>
	<title>{{cells["Title"].value}}</title>
	<subTitle>{{cells["Subtitle"].value}}</subTitle>
</titleInfo>
<name type="personal">
	<namePart>{{cells["Name"].value}}</namePart>
	<role>
		<roleTerm authority="marcrelator" type="text">{{cells["Role of Name"].value}}</roleTerm>
	</role>
</name>
<name type="personal">
	<namePart>{{cells["Name2"].value}}</namePart>
	<role>
		<roleTerm authority="marcrelator" type="text">{{cells["Role of Name 2"].value}}</roleTerm>
	</role>
</name>
<typeOfResource></typeOfResource>
<genre/>
<originInfo>
	<dateIssued>{{cells["Date"].value}}</dateIssued>
	<publisher/>
	<place>
		<placeTerm authority="marccountry"/>
	</place>
	<place>
		<placeTerm type="text"/>
	</place>
</originInfo>
<language>
	<languageTerm authority="iso639-2b" type="code">{{cells["Language1"].value}}</languageTerm>
</language>
<language>
	<languageTerm authority="iso639-2b" type="code">{{cells["Language2"].value}}</languageTerm>
</language>
<language>
	<languageTerm authority="iso639-2b" type="code">{{cells["Language3"].value}}</languageTerm>
</language>
<abstract>{{cells["Description"].value}}</abstract>
<identifier/>
<physicalDescription>
	<form authority="marccategory">sound_recording</form>
	<extent/>
</physicalDescription>
<note/>
<subject>
	<topic>{{cells["Topic1"].value}}</topic>
	<topic>{{cells["Topic2"].value}}</topic>
	<topic>{{cells["Topic3"].value}}</topic>
	<topic>{{cells["Topic4"].value}}</topic>
	<geographic/>
	<temporal/>
	<hierarchicalGeographic>
		<continent>{{cells["Continent"].value}}</continent>
		<country>{{cells["Country"].value}}</country>
		<province>{{cells["Province"].value}}</province>
		<region/>
		<county/>
		<city>{{cells["City"].value}}</city>
		<citySection>{{cells["City Section"].value}}</citySection>
	</hierarchicalGeographic>
	<cartographics>
		<coordinates>{{cells["Coordinates"].value}}</coordinates>
	</cartographics>
</subject>undefined</mods>

Row Separator

none

For Suffix

</mods>

Final Result

<mods
	xmlns="http://www.loc.gov/mods/v3"
	xmlns:mods="http://www.loc.gov/mods/v3"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns:xlink="http://www.w3.org/1999/xlink">
	<titleInfo>
		<title>Title of Oral History</title>
		<subTitle/>
	</titleInfo>
	<name type="personal">
		<namePart>Kim Day</namePart>
		<role>
			<roleTerm authority="marcrelator" type="text">Speaker</roleTerm>
		</role>
	</name>
	<typeOfResource>moving image</typeOfResource>
	<genre/>
	<originInfo>
		<dateIssued>2001-05-21</dateIssued>
		<publisher/>
		<place>
			<placeTerm authority="marccountry"/>
		</place>
		<place>
			<placeTerm type="text"/>
		</place>
	</originInfo>
	<language>
		<languageTerm authority="iso639-2b" type="code">fre</languageTerm>
	</language>
	<abstract>A description of the video goes here</abstract>
	<identifier/>
	<physicalDescription>
		<form authority="marccategory">sound_recording</form>
		<extent/>
	</physicalDescription>
	<note></note>
	<subject>
		<topic>cold</topic>
		<geographic/>
		<temporal/>
		<hierarchicalGeographic>
			<continent>North America</continent>
			<country>Canada</country>
			<province>Ontario</province>
			<region/>
			<county/>
			<city>Toronto</city>
			<citySection>Scarborough</citySection>
		</hierarchicalGeographic>
		<cartographics>
			<coordinates>43.787098, -79.185808</coordinates>
		</cartographics>
		<topic>winter</topic>
		<topic>city life</topic>
	</subject>
	<language/>
	<name type="personal">
		<namePart>Merle Haggard</namePart>
		<role>
			<roleTerm authority="marcrelator" type="text">Interviewer</roleTerm>
		</role>
	</name>
	<language>
		<languageTerm authority="iso639-2b" type="code">eng</languageTerm>
	</language>
</mods>

Ingest into Fedora

https://github.com/digitalutsc/fedora_video_ingesting

Ingest into Islandora through the interface - recommended file formats and sizes

Document type Content types Islandora accepted file formats Size restrictions *

---

Oral history, digital storytelling, interview audio recording wav, mp3 less than 200 mb video recording ogg, mp4, mov, qt, m4v, avi, mkv less than 200 mb

Converting tab delimited/text files created in Transcription Software into XML

The files created in transcription software must be converted into XML to make it possible for ingest into Islandora. The resulting file will be named TRANSCRIPT.xml.

Parts of this Section:

• Simple XML Transcript Template

• Converting a Tab Delimited File Created in Inqscribe into XML

  • Tools

  • Converting the timestamps to seconds in Excel

  • Steps to create XML from Inqscribe Tab Delimited File Using Open Refine

• Optional: Validating transcripts using Oxygen XML Editor

• Converting a Tab Delimited File created in Audacity into XML

Simple Transcript XML Template

The elements:

• cues

  • cue

    • speaker or solespeaker

    • Start

    • end

Tiers (optional):

• Transcript

• Translation

• Annotation

Tiers are the additional pieces of information that you want to associate with a start and end time and speaker in a transcript. Your oral history might have transcriptions only, so you would then add the <transcript> element into your TRANSCRIPT.xml. They might have transcriptions and translations in which case you would add both the <transcript> and <translation> element into your TRANSCRIPT.xml. If a tier exists in the transcript, they always need to appear in every cue and they always need to be in the same order.

If an oral history only has one speaker for the entire clip use only solespeaker. If there are multiple speakers, use the speaker element. Solespeaker and speaker should never appear together in the same transcript.

Example

<?xml version="1.0" encoding="UTF-8"?>
<cues>
	<!-- If the entire transcript has one speaker only, use 'solespeaker' element. Then skip 'speaker' element in 'cue' element level. But DO NOT use them in both places. If 'solespeaker' element presents in the document, following 'speaker' elements will be ignored. -->
	<solespeaker>One Speaker</solespeaker>
	<cue>
		<speaker>Different Speaker</speaker>
		<!-- 'start' and 'end' elements are start time and end time in seconds for the cue. -->
		<start>0.000</start>
		<end>12.124</end>
		<!-- 'transcript' and/or 'annotation' are default content tiers of the cue. Extra tier(s) can be added as long as they are listed in the configuration page. 'transcript' element is required if 'Enable captions/subtitles display' is configured to be true. -->
		<transcript>This is the transcript text content.</transcript>
		<annotation>This is the annotation content.</annotation>
	</cue>
	<!-- add more cues with above structure.-->
</cues>

Converting a Tab Delimited File created in Inqscibe into XML

Tools

There are two tools needed for this conversion

Excel

OpenRefine http://openrefine.org/

http://digitalscholarship.utsc.utoronto.ca/projects/content/blogs/converting-spreadsheets-modsxml-using-open-refine

Important: Use UTF-8 Unicode as your text encoding format! You can make sure you do this when you import the data into Excel.

Converting the timestamps to seconds in Excel

Only use these steps for Inqscribe Transcripts

The islandora oral histories solution pack reads time as seconds. Instead of 00:00:23.10 it would need 23.1. To change the timestamp follow these steps:

1. Open Excel then go to File> Open and select the transcript file you are working with and click open.

2. The Text Import Wizard will open. Click the next button until you get to step 3 of 3 then click Finish.

3. Remove all unicode encoded characters you see within the spreadsheet ex. Ôªø.

4. Rename the IN column START and the OUT column END.

5. Insert two new columns one following the IN column and one following the OUT column. Call the two new columns IN and OUT. This step can be seen below.

6. You will now convert the time in the START and END columns into seconds. To do this insert the formula =A2*86400 into the new IN Column (B2) and the formula =C2*86400 into the new OUT column (D2). See Figure (x) for example.

6. At this stage you will copy down the formula you just inserted. To do this click the cell where you just entered the formula. Then bring your cursor to the bottom right corner of that cell until it turns into a solid black cross and then double click. The format of the IN and OUT columns should match figure (x) seen below.

7. Save the file as a tab delimited file. Go to File> Save As... Ensure that tab delimited file is selected in the format box and click save.

Steps to create XML from Inqscribe Tab Delimited File Using Open Refine

(Only use these steps for Inqscribe transcripts)

1. Open OpenRefine

2. Click the Browse button and select the file you are working with and click Next>Create Project. Make sure the character encoding is UTF-8 in before creating the project.

3. Remove the START column and the END column with the time format 00:00.0. To do this click the down arrow in the header of the column title then click Edit column>Remove this column. See example below.

4. To begin the process of exporting the table to XML click Export> Templating. The Templating Export box will appear.

5. Copy and paste the text below into your the Templating Export box. Ensure that you match the appropropriate text to the input box using the bold headers as guides.

prefix

prefix

<?xml version="1.0" encoding="UTF-8"?>
<cues>

row template (IMPORTANT: make sure your headers in your transcript file match the headers in the row template!)

<cue>
<speaker>{{cells["SPEAKER"].value}}</speaker>
<start>{{cells[“START"].value}}</start>
<end>{{cells[“END"].value}}</end>
<transcript>{{cells["TRANSCRIPT"].value}}</transcript>
</cue>

Row template (Use this version If your transcript has tiers only)

<cue>
<speaker>{{cells["INTERVIEWER"].value}}</speaker>
<start>{{cells["IN"].value}}</start>
<end>{{cells["OUT"].value}}</end>
<transcript>{{cells["TRANSCRIPT"].value}}</transcript>
<translation>{{cells["TRANSLATION"].value}}</translation>
<annotation>{{cells["ANNOTATION"].value}}</annotation>
</cue>

row separator

Make this box empty, remove any characters

suffix

</cues>

Final Result

The resulting file should match this format:

<cue>
<speaker>Dorjee Lhatoo</speaker>
<start>36.0</start>
<end>45.0</end>
<transcript> Poor company with me. She would pour me a glass, one sip and it goes right through, you see -</transcript>
</cue>

<cue>
<speaker>Interviewer2</speaker>
<start>46.0</start>
<end>49.0</end>
<transcript> [indiscernible] on fire. She sounds like great fun.</transcript>
</cue>

<cue>
<speaker>Dorjee Lhatoo</speaker>
<start>49.0</start>
<end>52.0</end>
<transcript> And then she used to bidi</transcript>
</cue>

<cue>
<speaker>Interviewer2</speaker>
<start>52.0</start>
<end>53.0</end>
<transcript> Oh!</transcript>
</cue>

Converting a Tab Delimited File created in Audacity into XML

Follow these steps if you created a transcript using Audacity

1. Open OpenRefine

2. Click the Browse button and select the file you are working with and click Next. Find the box labeled “line(s) as column headers” located at the bottom right of the screen and change 1 to 0. Click the Create Project button located at the top right of the screen. See example below.

1. We now need to create a new column for speaker name. The speaker name information will be contained in column 3. To create the new column click the down arrow next to the column 3 header then click Edit column> Split into several columns…. A box titled split Column 3 into several columns. Change the Separator to : and click OK. An example can be seen below.

1. You now need to change the column headers. To do this click the down arrow beside the column headers then click Edit column> Rename this column. Enter the new name and click OK. Please see example below. You new column headers should be:

   • IN (Replacing Column 1)

   • OUT (Replacing Column 2)

   • SPEAKER (Replacing Column 3 1)

   • TRANSCRIPT (Replacing Column 3 2)

4. To begin the process of exporting the table to XML click Export> Templating. The Templating Export box will appear.

5. Copy and paste the text below into your the Templating Export box. Ensure that you match the appropropriate text to the input box using the bold headers. Then click the Export button. Give file the appropriate name and place in the appropriate directory.

prefix

<?xml version="1.0" encoding="UTF-8"?>
<cues>

**row template **

<cue>
<speaker>{{cells["SPEAKER"].value}}</speaker>
<start>{{cells[“START"].value}}</start>
<end>{{cells[“END"].value}}</end>
<transcript>{{cells["TRANSCRIPT"].value}}</transcript>
</cue>

Row Template (Use only If your have transcript has tiers, change this template to match what tiers you are using.)

<cue>
<speaker>{{cells["INTERVIEWER"].value}}</speaker>
<start>{{cells["IN"].value}}</start>
<end>{{cells["OUT"].value}}</end>
<transcript>{{cells["TRANSCRIPT"].value}}</transcript>
<translation>{{cells["TRANSLATION"].value}}</translation>
<annotation>{{cells["ANNOTATION"].value}}</annotation>
</cue>

row separator

none

suffix

</cues>

8. The resulting XML file should have the structure seen below

Final Result

<cue>
<speaker>Dorjee Lhatoo</speaker>
<start>36.0</start>
<end>45.0</end>
<transcript> Poor company with me. She would pour me a glass, one sip and it goes right through, you see -</transcript>
</cue>

<cue>
<speaker>Interviewer2</speaker>
<start>46.0</start>
<end>49.0</end>
<transcript> [indiscernible] on fire. She sounds like great fun.</transcript>
</cue>

<cue>
<speaker>Dorjee Lhatoo</speaker>
<start>49.0</start>
<end>52.0</end>
<transcript> And then she used to bidi</transcript>
</cue>

<cue>
<speaker>Interviewer2</speaker>
<start>52.0</start>
<end>53.0</end>
<transcript> Oh!</transcript>
</cue>