Subs browser for Stanford ai-class

Oct 17

Introduction

So, last week Stanford ai-class has started.

Immediately found out that not being able to read question for quizzes in text was a problem. And clicking trough video to see it in the subs was not that fun aether. Immediately got an idea to fix that for myself and make an exercise in HTML/JS/CSS out of it (I am kind of slowly learning it all trough small projects like this for last year).

Try number one: bookmarklet

First, I remembered of bookmarklets. Small javascripts saved as a bookmark which you can “execute” inside any page. Developers often put in some small useful codes to inject in to pages for analysis or some other reasons.

After checking ai-class page and requests found out from where player gets subs. Whole link is large and contains lot of unnecessary parameters, experimenting with it shrunk it down to smaller size. And here how it looks.

Then googled around a little for ways to make a cross domain request from JavaScript. Thing is that for security reasons in JavaScript making such requests is prohibited while for Flash a cross domain policy exists that allows domain owners to specify who can gain access. Found out that there is a partial workaround trough Yahoo! YQL.

In the end I failed with that idea as there is no way to really figure out which video is playing right now on ai-class site. At least trough JS/HTML. May be it is possible to somehow query YouTube player for that.

P.S Killed lot of time on hacking 3 levels of escaping of quotes. First quotes were for anchor href attribute in which bookmarklet code goes. Then quotes inside that code. And third quotes level in that YQL request I hate escaping!

Try number two: php curl

Next quick try was to test PHP curl and see if I can retrieve subs from those urls on server side and make a page out of it. Tried, failed, after lot of fighting with JS on previous step gave up fast. And I don’t have idea how to “debug” curl http requests on server side anyways.

Try number three: Flash local app

Well after some fighting with JS and PHP I fall back to dark side I am Flash developer for last 5 years soo… Something like an hour and I got this:

Just an app where I paste link to AI-Course video and get their subs formatted for reading.

Until homework started this was enough for me.

Try number 4: YouTube JS Api + XML

While doing home works I realized that only text for current video is not enough. Seeing people making lecture notes helped that realization. So I needed ability to get all subs together and search trough them. But wait, text is not enough. Jumping to other videos I need to quickly refresh my knowledge is a “must” too.

That’s how idea started to form, I rewrite my app to take in all videos and subs for them. Output it in one XML. Use it as data source for a page. And add YouTube player controlled from JavaScript to the mix.

First things I went to check was how to get list of ai-class lectures automatically. I am a programmer and for me doing copypasting job is a disgrace. I better write a script to do it for me Poked around Zen GData library for PHP and its YouTube API. Turned out it was not so good for what I needed and there was easier way just trough links like this. Sadly it only lists public videos, so no homework’s. And also order is mixed, ended up parsing video names for ordering. Also turned out that only video owner can get subs trough their API So my hacky way of spoofing player requests seems to be the only way.

Anyways, rewrote my Flash app to get all videos from there, sort them, get subs for each trough old method and output an XML like this. Also ended up adding homework video ids by hand.

And then started making site:

Made a PHP page with YouTube player and scripts to control it. Fast and easy.
Used SimpleXML in PHP to parse XML and output a list of lectures with subs. Easy again.
And then part I don’t like about current standard web development came, layout trough CSS. It got lot better in last 5 years but its still feels like hacking around achieving what you need trough not intuitive means. That’s why Flash won me over 7 years ago

P.S I actually love CSS selectors idea, just layout part feels broken. And I love jQuery for bringing selectors to JS.

Result

And here is the result:

You can open or close lectures separately. Or close/open them all together.
You can read whole transcripts of lectures and search trough them trough standard browser means.
And you can click on button to the right of each line to jump to that video on that moment. Magic!

There are still some problems though:

Firstly, using lists for layout here seems to be a bad idea. Should have used table with two columns instead. Will try soon.
There are some issues under Mac/Safari with referencing YouTube Player. As I am under winds can try stuff only blindly.
And script for making an XML still is in Flash/AS3, will need to spend some time trying to port it to PHP, not sure it will work.

Source code

Ok, you can read more about source code in next post or go straight to GitHub repository where it is.

none

Dan

Great job Eduard, really useful.
a fellow ai-class student

http://www.wonderwhy-er.com wonderwhy-er

Hey Dan, thanks and glad it helps
Will update it today, few fixes + new lectures and homework’s + am finally planing to share source code

Fellow student

I think you should tweet this to aiclass. Good job

http://www.wonderwhy-er.com wonderwhy-er

Thanks, I thought I did yesterday https://twitter.com/#!/wonderwhy_er/status/128937867358650372
Seems that it just was a tween with mention. Well I am not much of twitter user and only registered as it seems to be only communication channel with people behind the course currently.

Dave

This is an absolutely indispensable resource for the AI Class. THANK YOU!

If the experience of Stanford’s server failures over the last two weekends is any guide, I imagine that it would fall right over under the crushing load of 160,000 students, but I wonder if our professors wouldn’t like to know about it, possibly promote it on ai-course.com? Perhaps you can get someone to host it on a beefy platform, to avoid killing your host?

Here’s a small suggestion: can you add another level of hierarchy so that it’s possible to collapse entire units? Thankfully, the consistent naming of the videos (even if they are out of sync with the numbering on the outline at https://www.ai-class.com/home/) means that you could pull the groupings from the data — I see that you’re trying to do this in as automated a manner as possible.

http://www.wonderwhy-er.com wonderwhy-er

Haha, thanks

Hmm, so far its not even close to big load on this side. Also I suppose their problems are mostly on tests/progress side, not static pages. Anyways, I will see what to do if it will ever come to problems. May be will switch to google app engine for a test

On a suggestion. It is coming soon, was thinking about adding another nesting level since 5-6 units were added. It did become too much to navigate this as it is. I just did not got to it yet.
But it will just take first part of titles. Unit 1, Unit 2, Homework 1 etc and group by them in to one more nesting level.

Compuneo

Very useful. I am happy to tweet your effort at https://twitter.com/#!/vijayk/status/141645643667079168

http://www.wonderwhy-er.com wonderwhy-er

Thanks, glad it is of help Helps me a lot too.

student

Thanks so much Eduard. This was really helpful. I’ve just managed to find enough time to listen to lectures and finish homework, so I really have had much time to print up material from this course so I’ll remember some of it weeks/months from now. It would be great if you would leave this site up after the course.

http://www.wonderwhy-er.com wonderwhy-er

Welcome.
I am not planing to remove it, but it will not be updated after I finish with the course.
And future version of the course will probably be different in some ways… So I guess with time this page will become meaningless.

zivagirl

Eduard it is truly amazing, thank you so much for this unbelievable effort. I am not certain how you had time to do it, but thank you!!!

http://www.wonderwhy-er.com wonderwhy-er

Thanks
Well its not unbelievable effort, though it did took me some time. But I don’t know how much time I would have spent searching manually trough lectures to repeat some material and trying to read/hear quizz questions if I did not make this

Jagoire87

I think you should tweet this to aiclass. Good job

http://www.wonderwhy-er.com wonderwhy-er

They actually mentioned this thingy on their main page few weeks ago, so yeah they are aware of it

Thankful student

Thank you so much for sharing. Your web page helped me a lot as I attended the course. Now that it is over, do you think you could update it one last time to capture subs from the answer videos to the final exam?

http://www.wonderwhy-er.com wonderwhy-er

Welcome
Updated today, it seems not all answers have subs yet. Will update once more soon.

wesey E batton

So informative post!
Stanford ai class is awesome they provides the projects for the student related HTML,JAVA and PHP etc.Actually ai class is much beneficial but the time has been passed if this ai class get again started .we will definitely join this .I have heard one thing regarding this type of project . They get created more viruses on your computer it’s Browser Security is must .To protecting your operating system.