Tuesday, August 14, 2012

Reviving the kobo touch

A couple of weeks ago my wife's kobo touch decided to die. Actually I think this happened because she didn't use it for a while. I took it to load some books, connected it to the PC, loaded the books and when I pulled it out it froze on the "Connected and charging" screen. I started roaming the forums to try and find a solution for this: first I left it to charge on the usb port, then I tried with a wall charger (bought for the kobo from fnac). I left it for over 12 hours with no result. When trying to start it the led flickered blue briefly (under a second if not plugged in) and did nothing after that. When connected to the computer or to the wall charger the led was green (charging) for about 10 seconds, went blue (assume trying to start) for about 5 seconds but didn't start. It went on doing this indefinitely. I did try to perform a factory reset using both methods: keep menu button pressed while powering on or keeping the back reset button pressed while powering on. This achieved nothing. The led just briefly flickered red twice. Following some advice on the forums I took off the back cover and disconnected the battery. Same story after connecting again so I thought there might be something wrong with the device. I opened up my own (working) device and connected the battery to the dead one. It started up immediately. I gave it up for a dead battery and thought I'd talk to a fried about finding a replacement and went on holiday. If you are interested in how the kobo came back from the dead please read on. As luck has it I didn't take my phone charger with me (it has the same type of connecter as the kobo) as I had the kobo wall charger and thought I'd use that. On the first day my phone battery died so I plugged it into the kobo charger but, surprise, it just stood there. I tried all the plugs in the room but with no success. Finally I just took the cable from the charger and asked the lady at the reception desk to plug the phone into their PC which worked and my phone returned to life. This got me thinking so when I got home I plugged the dead kobo touch into my phone charger, left it for about an hour. This time the green led stayed on all the time (no blue light). After an hour I tried to switch it on: the led wen from green to blue for about 5 seconds and then back to green. I was about to throw it at the wall in frustration when the screen turned white and the device powered up. My assumption is that the battery was fully depleted and, for some reason, the device tries to switch on even when connected to the wall charger (either this or the charger doesn't provide enough power) which drains the battery again so it never has enough time to store the power needed for startup. This is probably an engineering flaw as I don't see the point of trying to start the device when there isn't enough power. It should probably only try to connect to the PC when the device is actually turned on. So, before sending your touch back to kobo try a different charger. I used a samsung phone charger. Hope this helps someone. I'll keep it for future reference just in case...

Thursday, July 19, 2012

Koboish ebooks

I really like the kobo touch but I noticed that there is a difference in how side-loaded epubs and epubs downloaded from the kobo store are treated by the device. The thing that I liked most about the ones from kobo was the page numbering that corresponds to actual screens required to reach the end of a chapter. After noticing this I started combing the net and discovered the infamous kepub format. On finding this I also found that changing the extension of the files from .epub to to .kepub.epub should give me access to some of those features so I did just that.
Results were mixed:
  1. Hooray, we've got page numbering
  2. In addition to page numbering we can also control the text appearance  using the fonts menu
  3. hmmm, where did the covers go? (actually I did read about this happening)
  4.  the third thing is more subtle and I only noticed that after reading about 30 pages of a chapter, closing the book and then trying to pick up where I had left off: while the chapter was correct, it always loaded the first page. The problem here is that I could no longer add annotations to the book.
Since I really, really wanted that page numbering but I don't have the time to read a book in one go I started to poke around a little.
A warning first:
  1. this work on the kobo touch with the 2.0 firmware
  2. While I've managed to get covers and annotations working I don't know if anything else broke
  3. FOLLOW THIS GUIDE AT YOUR OWN RISK
The first thing you need to do is to change the extension of your epub to .kepub.epub and upload it to your kobo (disconnect the device to trigger it to detect the book). The rest I'll split into three parts:
Give me my cover back
Recover annotations
Get lazy with scripting


Give me my cover back

 The first thing I wanted to do was get rid of the ugly kobo generated covers. Even if the epub has a cover set after changing the extension to .kepub.epub the cover is no longer used and is replaced with some generated covers. 
In order to fix this you need to do the following:
  1. Assuming you cover is a jpeg file named mywonderfullcover.jpg
  2. you will need your cover in four sizes (you can also convert it to grayscale to reduce the size)  and each one of this files needs to have a special name and the extension changed from .jpg to .parsed:
    1.  "mywonderfullcover  - N3_LIBRARY_FULL.parsed" - size: 355 x 530 pixels
    2.  "mywonderfullcover  - N3_LIBRARY_GRID.parsed" - size: 149 x 223 pixels
    3.  "mywonderfullcover  - N3_LIBRARY_LIST.parsed" - size: 60 x 90 pixels
    4.  "mywonderfullcover  - N3_LIBRARY_SHELF.parsed" - size: 40 x 60 pixels
  3. Connect you kobo and copy the files you created to the .kobo/images folder 
  4. Next you'll need to update the sqlite database. The file can be found at .kobo/KoboReader.sqlite. I suggest making a back of this first. Open the database. I use SQLiteSpy but you can use any sqlite browser you like. Open up the content table and look for the row that contains your book name in the ContentID column and has the ContentType column set to 6. The ImageId column for this row should be empty. Set it to "mywonderfullcover".
  5. Save the changes and disconnect the reader
  6. In order for the cover to be detected you have to restart the reader (if anyone knows of another way of refreshing the data from the database please let me know) 
Now you should have your cover back, plus the additional benefits already listed but you still don't have any annotations


Recover annotations

At first I thought that the annotations are kept somewhere in the database...I was wrong.
I also tried putting the file in the .kobo/kepub folder and adding the required entried manually in the database but still there were no annotations  although I did learn some stuff about the database structure while doing this
After doing a little more reading on the forums I found some opinions that there are some special javascripts involved so I opened a free book that I got from the kobo site. This also proved wrong but it did lead me in the correct direction.
There doesn't seem to be any javascript involved but, in looking at the html files it look like the html files inside the kepubs have some special tags inserted that allow annotations to be retrieved. 
So what I did was open each html file and wrap the content of each top level h and p tags in a span tag with the id set to "kobo.[incremental_count]".1 where incrementa_count starts at 1 and goes as high as you need it to.
To illustrate this let's suppose you have the following insisde the body tag of your html:
<h1>Chapter</h1>
<h2>SubChapter</h2>
<p>some paragraph</p>
<p>some other paragraph</p>
<p>the last paragraph</p>

This would be changed to:
<h1><span id="kobo.1.1">Chapter</span></h1>
<h2><span id="kobo.2.1">SubChapter</span></h2>
<p><span id="kobo.3.1">some paragraph</span></p>
<p><span id="kobo.4.1">some other paragraph</span></p>
<p><span id="kobo.5.1">the last paragraph</span></p>

Upload your .kepub.epub back on the device and enjoy the freshly recovered annotations.


Get lazy with scripting

 

Automatic renaming using calibre

If you use calibre you can configure it to rename your file when uploading it to the reader by going to "Preferences->Sending books to device" and  and adding ".kepub" at the end of the title template.


Automatic photo shop script for updating images

If you use photoshop you can use the script below to automatically create and rename your cover files (save the code in a file with the extension set to .jsx). Just open your jpeg file in photoshop and go to  File->Scripts->Browse and select the file where you saved the script code. Your original file will be left unchanged.
    #target photoshop
    main();
    function main(){
    if(!documents.length) return;
    var startRulerUnits = app.preferences.rulerUnits;
    app.preferences.rulerUnits = Units.PIXELS;
 var quality = 10
    var doc = app.activeDocument;
    var Name = doc.name.replace(/\.[^\.]+$/, '');
    var Path = decodeURI(doc.path);
 var outFolder = Folder(Path);// +"/"+Name);
    if(!outFolder.exists) outFolder.create();

 doc.changeMode(ChangeMode.GRAYSCALE);
 doc.bitsPerChannel = BitsPerChannelType.EIGHT
    createNamedSnapshot("Snap 1");

    doc.resizeImage(355, 530, 96, ResampleMethod.BICUBIC);
    var saveFile = File(outFolder +"/"+Name+" - N3_LIBRARY_FULL.parsed");
    SaveJPEG(saveFile,quality);
    var saveFile = File(outFolder +"/"+Name+" - N3_LIBRARY_FULL.jpeg");
    SaveJPEG(saveFile,quality);
    revertNamedSnapshot("Snap 1");
 
    doc.resizeImage(149, 223, 96, ResampleMethod.BICUBIC);
    var saveFile = File(outFolder +"/"+Name+" - N3_LIBRARY_GRID.parsed");
    SaveJPEG(saveFile,quality);
    revertNamedSnapshot("Snap 1");
 
    doc.resizeImage(60, 90, 96, ResampleMethod.BICUBIC);
    var saveFile = File(outFolder +"/"+Name+" - N3_LIBRARY_LIST.parsed");
    SaveJPEG(saveFile,quality);
    revertNamedSnapshot("Snap 1");
 
    doc.resizeImage(40, 60, 96, ResampleMethod.BICUBIC);
    var saveFile = File(outFolder +"/"+Name+" - N3_LIBRARY_SHELF.parsed");
    SaveJPEG(saveFile,quality);
 
    app.activeDocument.close(SaveOptions.DONOTSAVECHANGES);
    app.preferences.rulerUnits = startRulerUnits;
    }

    function SaveJPEG(saveFile, jpegQuality){
    jpgSaveOptions = new JPEGSaveOptions();
    jpgSaveOptions.embedColorProfile = true;
    jpgSaveOptions.formatOptions = FormatOptions.STANDARDBASELINE;
    jpgSaveOptions.matte = MatteType.NONE;
    jpgSaveOptions.quality = jpegQuality; //1-12
    activeDocument.saveAs(saveFile, jpgSaveOptions, true,Extension.LOWERCASE);
    }

    function createNamedSnapshot(name) {
        var desc = new ActionDescriptor();
            var ref = new ActionReference();
            ref.putClass( charIDToTypeID('SnpS') );
        desc.putReference( charIDToTypeID('null'), ref );
            var ref1 = new ActionReference();
            ref1.putProperty( charIDToTypeID('HstS'), charIDToTypeID('CrnH') );
        desc.putReference( charIDToTypeID('From'), ref1 );
        desc.putString( charIDToTypeID('Nm  '), name );
        desc.putEnumerated( charIDToTypeID('Usng'), charIDToTypeID('HstS'), charIDToTypeID('FllD') );
        executeAction( charIDToTypeID('Mk  '), desc, DialogModes.NO );
    }

    function revertNamedSnapshot(name) {
        var desc = new ActionDescriptor();
            var ref = new ActionReference();
            ref.putName( charIDToTypeID('SnpS'), name );
        desc.putReference( charIDToTypeID('null'), ref );
        executeAction( charIDToTypeID('slct'), desc, DialogModes.NO );
    }

 

Automatic kobofying

Now all that is fine but what if you don't want to spend twice as much time as is requred to read a book just to kobofy it.
No worries, since I'm a little lazy myself I create the small python script bellow. In order to run it you need python 2.7 and BeautifulSoup installed. 
In order to use it unpack your files to a folder, create file called something.py inside that folder containing the code bellow and run it by calling "python something.py". 
Warning: this script was hastily put toghether so it has a few restrictions:
  1. It always adds the spans to h and p tags so don't run it multple times on the same file 
  2. It only works on top level h and p tags so, for example, if your h and p tags are wrapped in a div or another tag it won't work.

import sys
from bs4 import BeautifulSoup
import re
import os, os.path
 
def altertags(soup):
 counter = 1
 for tag in soup.body.find_all(re.compile("^(p|h)"), recursive=False):
  new_tag = soup.new_tag("span", id="kobo."+str(counter)+".1")
  counter = counter + 1
  tag.wrap(new_tag)
  tag.unwrap()
  new_tag.wrap(tag)

def parsefile(infile, outfile):
 soup = BeautifulSoup(open(infile))
 altertags(soup)
 output = open(outfile,'w')
 output.write(str(soup))
 output.close()
 

if (len(sys.argv) > 2):
 infile = sys.argv[1]
 outfile = sys.argv[2]
 parsefile(infile, outfile)
 
if (len(sys.argv) == 2):
 infile = sys.argv[1]
 parsefile(infile, infile)

if (len(sys.argv) < 2):
 for file in os.listdir("."):
  if file.endswith("html"):
   parsefile(file, file) 
If someone can improve this please share.
I don't actually use this as it requires you to unpack the book, repack and so on and that's to much work for me so I just modified kiwidude's Modify Epub plugin for calibre to do this for me.

Well...that's about it. If anyone reads this and finds it useful feedback is always appreciated.