Amara API Switchover

The new API code has been running in production for a good while now without too much problem.  We’re going to switch to using that code only starting on our next deploy which will probably happen tomorrow.

The old URLs will continue to work, they are just going to be handled by the new code rather than the old code.  I previously talked about having them redirect to the new URLs, but it’s simpler to skip that part and just return a response using the new code.

So what’s next for the API code?  There’s a couple directions that we’ll be going in and some are fairly exciting:

  • We’re going to work with our partners to change the partner-specific API endpoints to using the new code.
  • This isn’t API-related exactly, but once we are done with the old code there are a lot of upgrades we can make to our third-party libraries.  This is awesome for us amara devs and also will hopefully come with some performance improvements.
  • Continuous improvement.  One of the best things about the new API code/system is that it has a reasonable way to make changes to the API.  It allows for a transitional time where API clients can continue using the old interface, but also optionally using the new interface.

amara release notes 2015-05-04

Changes deployed to amara.org May 1, 2015

  • 449: Improve time calculation when inserting subs lines
  • 1291: Sync history should display to team admins
  • 1743: Give error message when “start syncing” is unavailable
  • 2166: make sure video URLs are found whether their URL is http or https
  • 2167: Switch to docker-compose
  • 2181: Syncing to YouTube overwrites ASR subtitles instead of creating a new track
  • 2182: ASR subtitles get automatically imported from YouTube
  • 2183: Project initialization from docker-compose
  • 2184: Add Sicilian language
  • 2189: New API returns null language in team video activity
  • 2190: Allow API GET requests for usernames with spaces

Changes deployed to amara.org April 24, 2015

  • 1967: Add Gothic language
  • 2092: Switch to youtube captions API v3
  • 2150: Post-publish metadata edits for TEDTalks do not send callbacks to TED
  • 2160: Send Permissive CORS headers with the new API
  • 2162: Some users cannot access their Messages | Sent Messages pages
  • 2168: Password reset form is gone from Django CustomUser form
  • 2169: Allow instant navigation down the subtitle set with ALT+Down keys
  • 2170: Replace ALT with Option for Mac OS X in keyboard shortcuts legend
  • 2171: ALT+i does not work on Mac OS X
  • 2173: Fixed password fields on user edit form (#2168)
  • 2183: Project initialization from docker-compose

YouTube Automatic Captions Need Work: A Chat with Michael Lockrey

Michael Lockrey is Deaf, which means he depends on high quality captions for the videos he watches on the web.

Unfortunately, many YouTube videos that are captioned use automatically generated captions. Automated captions are often inaccurate and, uncorrected, they can do more harm than good. This is partially due to the perception creators have, when they think automated captions are “good enough”. In light of this situation, Michael has taken a hands-on approach to making change.

Michael built a website and tool called nomoreCRAPTIONS to challenge the prevalence of automatic captions (or “craptions” as Michael likes to call them). nomoreCRAPTIONS is a simple tool for improving automated captions, but it’s also a campaign to raise awareness about the importance of accurate captions for all web videos.

Screenshot of nomoreCRAPTIONS website.We had a chance to catch up with Michael and learn more about his efforts:

======

Amara: How do you explain nomoreCRAPTIONS to people?

Michael: nomoreCRAPTIONS was designed to be the easiest way to fix up the automatic craptions on shorter YouTube videos (i.e. up to 2 or 3 minutes in duration).

We’ve deliberately focused on leveraging the intrinsic value in automatic craptions as it’s usually quicker to fix up these auto-generated craptions rather than starting from scratch.

The other main benefits of nomoreCRAPTIONS are that:

There’s no barriers or learning curve – you simply plug in the link to the YouTube video you want to fix up. This means that it’s very easy for anyone* to roll up the sleeves and fix up the errors in the auto-generated craptions.

* (anyone === who can hear)

You don’t have to be the content owner and this means that you don’t need to wait days / weeks / months (or even years) to watch the captions (as any corrections are instantly available on the nomoreCRAPTIONS website).

Amara: What motivated you to start the project?

Michael: As a Deaf man, I need good quality captions to watch online video content and it’s well known that YouTube is the biggest repository of online videos.

In the past if I’ve come across a viral or otherwise interesting video on YouTube that I’d like to watch – invariably it won’t have good quality captioning and I’ve been unable to do so.

YouTube has admitted recently that only 25% of YouTube videos have captioning and most of these only have automatic craptioning, which doesn’t provide me with any accessibility outcomes and I wrote a blog post recently that suggests that this means that only 5% of YouTube videos are likely to have good quality captioning and this simply isn’t good enough.

So you could say that nomoreCRAPTIONS has also become a bit of a personal protest against Google and YouTube’s over-use of automatic craptions and perhaps also a “cry for help” for them to do more.

Amara: How do you underline the importance of nomoreCRAPTIONS?

Michael: The lack of captioning on YouTube videos and other online video platforms is one of the biggest accessibility issues on the web today.

But it’s very hard to get this message through to the general public and as the internet is largely an unregulated environment it means that the advocacy strategies that made captioning available on almost 100% of traditional media such as TV in the USA, UK and Australia will not necessarily be effective as more and more content shifts onto the internet.

As a result I’m a very strong believer that we have a huge fight on our hands to gain accessibility to online video content and the best way to achieve this is by making it simple and easy to add good quality captioning.

Amara: Do you have a favorite story about nomoreCRAPTIONS? Any exciting impact that you’ve seen?

Michael: My favourite story to date has been working closely with my father on YouTube captioning using the nomoreCRAPTIONS tool.

He’s retired and he’s also lost quite a bit of hearing as he’s now well into his 70’s which means that rather than him and I being some sort of dynamic captioning duo akin to an #A11Y Batman and Robin, we’re probably have more in common with the two guys from Dumb and Dumber!

But he’s really enjoyed using the nomoreCRAPTIONS tool and he’s now even volunteering to provide the captions for the Australian War Memorial YouTube channel as he has a lot of family members who served in previous wars and he’s very interested in learning about the family history etc.

It was also great to work with Amara and many of their volunteers recently on an advocacy project that is trying to get the New York Times to caption their online videos.

Check out our public Trello project page and whilst it doesn’t use nomoreCRAPTIONS (as the New York Times has actually disabled the automatic craptioning on their YouTube channel) we have used an innovative workflow where we focus on creating an accurate text transcript first and then upload this onto the web where it is synced to the audio track.

Many of the volunteers have raved about how it’s so good to not have to manually sync the text to the audio track and it’s a significant saving of 20-30% of the standard captioning workflow time, which means that volunteers can create even more accessibility outcomes.

Amara: Where would you like to see nomoreCRAPTIONS go in 2015?

Michael: The best thing that could happen in 2015 for nomoreCRAPTIONS would actually be a nail in our coffin – we want Google and YouTube to cease publishing the automatic craptions immediately.

Instead we’d love to see Google and YouTube be far more proactive on ensuring that captions happen, particularly for the bigger channels and content producers, such as the New York Times.

For example, they could easily prompt users whilst they’re waiting for videos to upload onto their channel to make sure their videos are captioned or they could even ask users to caption the first 15 – 30 seconds of a video, etc.

Another thing I’d like to see is for much more “low hanging fruit” to be captioned and it would be great to see some “quick wins” up on the A11Y-scoreboard.

By this I’m simply referring to videos (such as those by the Australian Prime Minister, Tony Abbott, where there’s already an accurate transcript published on the Australian Government’s website and it’s also usually added to the meta data on their YouTube video page) but the videos themselves are still not captioned.

Check out the links here:

Screenshot showing the full transcript in the YouTube video’s meta-data section:

Screenshot showing the full transcript in the YouTube video's meta-data section:

TEDx Talks are another good example – as most presenters would have a written script, that can be easily uploaded and automatically converted into a very accurate caption file.

Amara: Are there ways people can help with nomoreCRAPTIONS and/or get involved in the larger captions/accessibility movement?

Michael: nomoreCRAPTIONS is an open source project and we are currently receiving some great support and mentoring assistance from the team at Free Code Camp. This means that we have some students working on the next version of nomoreCRAPTIONS with node.js support and numerous other improvements.

Our product roadmap for nomoreCRAPTIONS also includes some new tools that will make it more efficient for captioning longer videos (as we usually don’t recommend using the existing nomoreCRAPTIONS tool for videos of more than 3 minutes in length).

======

We’re very grateful to Michael for sharing his time and thoughts – we’ll continue to stay in touch and bring updates. If you’d like to learn more about his efforts and/or get involved with nomoreCRAPTIONS, please check out the links below.

amara.org production deploy 2015-04-15

Changes deployed to amara.org April 15, 2015

  • 1866: Editor: Don’t allow saving if timestamps are invalid
  • 1873: Add some more options per subtitle line in the editor
  • 2153: Editor shortcuts
  • 2157: Increase required version of coverage to fix warnings
  • 123: generalize editor-nonces / open-editor

Changes deployed to amara.org 21-March -> April 8, 2015

  • 1425: For YouTube videos, indicate whether they sync subtitles to YouTube
  • 1481: Editor: do not allow SHIFT+SPACE play and pause the video
  • 1950: Do not show line breaks in the transcript viewer
  • 1953: Upgrade to Django 1.4.20
  • 2047: Billing record not created for subs uploaded via the editor
  • 2112: Amara player displays subtitles with a delay
  • 2117: Display on diffing page how subtitle has changed – text, time, or both
  • 2126: Allow method to create a new user with a unique username
  • 2128: Track users created by other users via the API
  • 2146: Languages section in user profile is shown empty when it has data
  • 120: Duration and original lang from Pro Request not always stored on the video

The New API is Here

The new API is here!  Last week we deployed the last bits of new API code for the general API.  Partner-specific stuff still needs to be implemented, but now we’re going officially deprecate the old API and I’d like to take this opportunity to politely nudge all API client to move over to using the new code.  Here’s the details:

  • Starting today the old API is officially deprecated
  • All API clients are encouraged to move to the new API.
  • 3 weeks from now (4/28) the old API urls will start redirecting to the new API urls.  The actual date might be a couple days later, but it won’t be before 4/28.
  • The client code changes should just be changing URLs, but please read my last blog post for details: http://about.amara.org/2015/03/25/api-refactor-finishing-up/
  • The URLs are switching from
    https://amara.org/api2/partners/[path]

    to

    https://amara.org/api/[path]

 

API Refactor finishing up

I just pushed the last bits of code to re-implement the API endpoints.  This includes the team application, message, and activity endpoints.  So this means that all of the general API has now been re-implemented with new code (finally).  It’s pretty laughable in hindsight that I thought 2-4 weeks was a reasonable amount of time for this, given that it actually took around 9.  But I think we are finally in a position to start the transition in the somewhat near future.

The timeline is probably going to be testing and reviewing the documentation this week.  Then we will start transitioning next week.  “Start transitioning” means that we simply will say that the old API is deprecated and give API clients a few weeks to migrate to using the new URLs.  Hopefully for all clients, this simply will mean switching the URLs, since the APIs are almost exactly the same.  However, clients should definitely test to make sure that we didn’t accidentally change functionality for them.

Speaking of keeping the same functionality, when re-implemented the API I tried to change as little as possible, but some things did end up changing.  The main reason for this was either the documentation was unclear about what the intended functionality was, or I thought that a certain piece of functionality was simply a very bad idea.  These are the main areas of change that I know about:

  • Tasks: it was pretty unclear to me how the interface for PUT/POST was intended to work.  I tried to keep it working more-or-less the same, but some of the code in the old API seemed to be bugs to me so I didn’t translate them over.
  • Team delete: I removed the ability to do a DELETE request on teams.  This seems like something that should never really happen in practice.  If you are depending on this functionality, please send me an email and we can make things work for your use-case.
  • Blank/null values: There were several places in the code where blank strings, null values, the string “null”, and missing values were used interchangeably.  I tried to follow the conventions that the old API used as much as possible, but I can’t say that I got it 100% correct.  Please email me if it’s not working for one of your use cases.

Production deploy 2015/3/20

Changed deployed to amara.org 2015-03-20

129: Add DOIT team to community page
1783: notes are only preserving the first line break
2043: Show endorsements/send backs in the editor notes
2044: enable communication between collaborators in the editor
2047: Billing record not created for subs uploaded via the editor
2064: Switch to OpenID Connect for google
2089: amara editor doesn’t use the primary url
2114: Add way to bulk-sync youtube subtitles for users