5/15/2010

HearItLater - An Audio Helper For A Lazy Dog

I've blogged about what I wanted to do: An Audio Helper For A Lazy Dog

Well, I have implemented this thing. I call it HearItLater, but just for me - I have no motivation to market it. Here are some details, and of course some code to play with.

First of all, what you need:

  • an account at ReadItLater (http://readitlaterlist.com). I am a fan of this little helper, so I have built this little helper around this other little helper for me
  • some active entries at ReadItLater - I always have some, so I just don't check for if the current list is empty
  • an own appkey at ReadItLater - I have got my own for my own purposes
  • Windows box to run the unchanged script. The script was developed on Windows 7, feel free to migrate it in whatever direction
  • ActivePerl 5.10.1 for Windows. I used the 64bit version without any extensions except the auto-downloaded ones - out of the box
  • eSpeak (http://espeak.sourceforge.net) - the TTS software I used to produce WAV files out of the text files
  • a lot of motivation and patience to accept a silly robot's voice
Ok, now how it works:

  • it reads and iterates current your ReadItLater list
  • for every url in the list - if it's HTML - it downloads the content
  • it strips the content to the plain text
  • it creates a WAV file out of the stripped text
What are the known issues:

  • sometimes, eSpeak crashes for whatever reason
  • the reading quality is bad. Well, you understand it, but since the eSpeak command line tool doesn't use MS SAPI, it's really bad - for me. But much better, than nothing. And you can exeperiment with the parameters to find the adequate speed and pauses and so on which better fit your ear, as well as voices
  • I didn't invest much time in error checking in the script, so expect some surprises
  • the HTML page gets stripped completely, that means that every single link text and so one gets read without any system or concept. That's the trade-off
  • I don't take over original file names which could be extracted from the url. I just count the list items from ReadItLater in a loop and use the counter to name the files
  • Some WAV files are corrupted, I don't know why yet

Good, the rest seems to work. And here is the code - replace the placeholders (|...|) with your own corresponding stuff and enjoy the acoustic channel full of B-movie robot voices hammering unsorted information into your head while you do your normal job:



require HTTP::Request;
require LWP::UserAgent;
use JSON;
use HTML::TreeBuilder;
require HTTP::Headers;

$appkey = '|readitlater appkey|';
$user = '|readitlater user|';
$pass = '|readitlater password|';
$baseurl = "https://readitlaterlist.com/v2/";
$basepars = "?username=$user&password=$pass&apikey=$appkey";
$out = "|output path|";
$espeak = "|espeak install dir|\\command_line\\espeak.exe";
$espars = "-v en-us+f2 -s 180 -g 10mS";

$json = readitlater("get");
@urls = @{processJSON($json)};
$i = 1;
foreach $url (@urls) {
$txt = page2text($url);
text2wav($txt, $out, $i);
$i = $i + 1;
}

# readitlaterlist.com API connector
sub readitlater {
local $fun = $_[0];

return typed_wget("$baseurl$fun$basepars", '?'); #'?' = doesn't matter which content type
}

# wget, which can control the content type
sub typed_wget {
local ($url, $ctype) = ($_[0], $_[1]);

print "wget $url\n\n";

$request = HTTP::Request->new(GET => $url);
$ua = LWP::UserAgent->new;
$response = $ua->request($request);
if (($ctype eq '?') or ($response->headers->content_type eq $ctype)) {
return $response->decoded_content;
}
else {
return ''; # empty content if error of any kind
}
}

# JSON processor
sub processJSON {
local $json = $_[0];

$perl = decode_json $json;
%hash = %{$perl};
$list = $hash{'list'};
%hash = %{$list};
@urls = ();
foreach $key (%hash) {
$val = $hash{$key};
%hash_sub = %{$val};
push(@urls, $hash_sub{'url'});
}

return \@urls;
}

# process an online document - wget it, but only if it's HTML and strip it to the plain text
sub page2text {
local $url = $_[0];

$content = typed_wget($url, 'text/html');
$tree = HTML::TreeBuilder->new;
$tree->parse($content);
$stripped = $tree->as_text();

return $stripped;
}

# save the stripped text to a text file and use eSpeak to create a WAV out of it
sub text2wav {
local ($txt, $out, $cnt) = ($_[0], $_[1], $_[2]);

$of = "$out\\f$cnt.txt";
open _F, ">$of";
binmode(_F, ":utf8");
print _F $txt;
close _F;

$cmd = "$espeak -f $of -w $out\\f$cnt.wav $espars";
system $cmd;
unlink("$of");
}


5/14/2010

An Audio Helper For A Lazy Dog

If you, like me, don't have enough time or eye focus to read all the blogs and online articles flying around, but really would like to get this information into your head, you will quickly look for audio helpers. Our ears are "free" most of the time, and the corresponding sense is free, too. Why not use this channel?

For me, it works. It's like background, it's there. Maybe not 100%, but more than nothing. And in most of the cases it's ok. And if I need more and it's really interesting, I will come back to it late and read it.

So, I wanted to get myself a Nabaztag, but I won't - it's a toy.

So, my new idea is this, and I'm sure it's not new, but I don't care: I will take eSpeak from here: http://sourceforge.net/projects/espeak/. I will write a Perl script which will take my web pages of interest, for example from my ReadItLater list (if I can connect to it), and create an MP3 over a WAV out of them. Then, I'll throw those MP3s onto my iPhone and I have my acoustic channel - web pages spoken by a robot, just for me. I can even avoid iPhone and stay with the WAVs on my desktop, whatever.

So, that's it. An audio helper for a lazy dog.