Create Website Thumbnails using PhantomJS

Hope you read my old article “Create WebThumb using LAMP”, it used for creating web thumbnails from Xvfb virtual framebuffer that is more time and resource consuming.

Shell Script with Xvfb try to open web browsers in real remote server to capture web-thumbs…

That whole setup might take a full day and image quality is very bad….

Thanks to PhantomJS

PhantomJS is a headless WebKit with JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG.

Simple Javascript example

console.log(‘Loading a web page’);
var page = require(‘webpage’).create();
var url = ‘https://www.svnlabs.com/’;
page.open(url, function (status) {
//Page is loaded!
phantom.exit();
});

Features:

– Headless Website Testing
– Screen Capture
– Page Automation
– Network Monitoring

Fork on Github https://github.com/ariya/phantomjs

PhantomJS is using WebKit for a real layout and rendering engine, it can be used to capture web page as screenshot or PDF files. PhantomJS can render anything on the web page, it can be used to convert contents in HTML/CSS, SVG and Canvas.

URL2PNG-URL2PDF-HTML2JPG-URL2IMAGE
URL2PNG-URL2PDF-HTML2JPG-URL2IMAGE
// Thumbnail of Webpage using webthumb.js

var page = require('webpage').create(),
    system = require('system'),
    weblink,
    fname;

if (system.args.length !== 3) {
    console.log('Usage: webthumb.js webURL filename');
    console.log('Usage: webthumb.js https://www.svnlabs.com svnlabs.png');
    phantom.exit(1);
} else {
    weblink = system.args[1];
    fname = system.args[2];    
    page.viewportSize = { width: 320, height: 480 };
    page.open(weblink, function (status) {
    if (status !== 'success') {
        console.log('Unable to open webpage!');
    } else {
        page.render(fname);
    }
    window.setTimeout(function () {
      phantom.exit();
    }, 3000);
    });
}

Download webthumb.js

# phantomjs webthumb.js https://www.svnlabs.com svnlabs.png
# phantomjs webthumb.js https://www.svnlabs.com svnlabs.pdf

You can even capture webpage as good quality PDF file 😉

Output:

svnlabs.png
svnlabs.pdf

You can also use examples/rasterize.js for same sort of functions 😉

# phantomjs examples/rasterize.js https://www.svnlabs.com/ svnlabs.png
# phantomjs examples/rasterize.js ‘https://www.svnlabs.com/’ svnlabs.pdf

rasterize.js – rasterizes a web page to image or PDF
render_multi_url.js – renders multiple web pages to images
technews.js – captures Google News as a PNG image

Create WebThumb using LAMP

We have to install opera/firefox  first on server.

Then install Xvfb for Xvfb virtual framebuffer

html2image.sh
……………………….

#! /bin/sh

/etc/init.d/xvfb start

export DISPLAY=:7

DISPLAY=:7 opera –remote ‘openURL(‘$1′)’ &

sleep 20

DISPLAY=:7  import -window root $2

/etc/init.d/xvfb stop

Uses:
# ./html2image.sh  https://www.svnlabs.com   webthumb/svnlabs.png

Open FireFox in Xvfb: firefox.sh
#!/bin/sh
mozilla-firefox -a firefox -remote ‘openURL(‘$1′, new-tab)’ || mozilla-firefox $1 &

Uses:
# ./firefox.sh https://www.svnlabs.com

Second attempt

# wget http://mirror.aarnet.edu.au/pub/opera/linux/1011/final/en/i386/shared/opera-10.11.gcc4.shared.qt3.i386.rpm

# rpm -ivh opera-10.11.gcc4.shared.qt3.i386.rpm

# yum install tcl-devel libpng-devel libjpeg-devel ghostscript-devel bzip2-devel freetype-devel libtiff-devel

# wget ftp://ftp.fifi.org/pub/ImageMagick/ImageMagick-6.6.2-7.tar.gz

# tar zxvf ImageMagick-6.6.2-7.tar.gz; cd ImageMagick-6.6.2-7

# ./configure –prefix=/usr/local –with-bzlib=yes –with-fontconfig=yes –with-freetype=yes –with-gslib=yes –with-gvc=yes –with-jpeg=yes –with-jp2=yes –with-png=yes –with-tiff=yes

configure: error: no acceptable C compiler found in $PATH

# yum groupinstall “Development Tools” -y

# ./configure –prefix=/usr/local –with-bzlib=yes –with-fontconfig=yes –with-freetype=yes –with-gslib=yes –with-gvc=yes –with-jpeg=yes –with-jp2=yes –with-png=yes –with-tiff=yes

# make clean

# make

# make install

Install xvfb
…………

# yum update

# yum install xvfb (No package xvfb available.)

http://corpocrat.com/2008/08/19/how-to-install-xvfb-x11-server-in-linux-server/

TRy with # yum install Xvfb
or # yum install xorg-x11-Xvfb

# yum install xorg-x11-fonts*

# vi /etc/rc.d/init.d/xvfbd (http://eldapo.lembobrothers.com/2009/05/27/an-xvfb-init-script/)

# chmod a+x /etc/rc.d/init.d/xvfb

# chkconfig –add xvfb

# /etc/init.d/xvfb start (start xBuffer)

Allow remote vnc for xBuffer
http://www.karlrunge.com/x11vnc/faq.html#faq-xvfb
http://wiki.centos.org/HowTos/VNC-Server
http://thelinuxguru.blogspot.com/2008/02/installing-x11vnc-on-fedora-core.html
http://wiki.centos.org/HowTos/VNC-Server

# wget http://space.dl.sourceforge.net/project/libvncserver/x11vnc/0.9.10/x11vnc-0.9.10.tar.gz

# tar zxvf x11vnc-0.9.10.tar.gz; cd x11vnc-0.9.10

# ./configure && make && make install

Error…

*** A working X window system build environment is required to build ***
x11vnc. Make sure any required X development packages are installed.
If they are installed in non-standard locations, one can use the
–x-includes=DIR and –x-libraries=DIR configure options or set the
CPPFLAGS and LDFLAGS environment variables to indicate where the X
window system header files and libraries may be found. On 64+32 bit
machines you may need to point to lib64 or lib32 directories to pick up
the correct word size.

If you want to build x11vnc without X support (e.g. for -rawfb use only
or for native Mac OS X), specify the –without-x configure option.

# rpm -q vnc-server

# yum install vnc-server

# chkconfig vncserver on

…………………………….

# wget http://dag.wieers.com/rpm/packages/x11vnc/x11vnc-0.9.3-1.el5.rf.i386.rpm

# yum install x11vnc-0.9.3-1.el5.rf.i386.rpm

………………………………

# yum install vnc

# yum groupinstall “GNOME Desktop Environment”

# useradd svnlabs; passwd svnlabs

#

At local linux box

# vncviewer -via sandeep@179.28.78.170 localhost:1

Other method

# /etc/init.d/xvfb status

# x11vnc -usepw -display :5

# ssh -C -t -L 5900:localhost:5900 [179.28.78.170] ‘x11vnc -usepw -localhost -display :5’

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

# DISPLAY=:5 opera –remote ‘openURL(http://www.yahoo.com)’ &
# DISPLAY=:5 import -window root /var/www/vhosts/trueweddingjunkie.com/httpdocs/sv.jpg

ERROR: ld.so: object ‘libjvm.so’ from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object ‘libawt.so’ from LD_PRELOAD cannot be preloaded: ignored.
/usr/lib/opera/opera: error while loading shared libraries: libqt-mt.so.3: cannot open shared object file: No such file or directory

ooooooooooooooooooooooooooooooooooooooooo

It means that java plugin was not found.

ooooooooooooooooooooooooooooooooooooooooooooo

Installing Java
……………

http://rahulsoni.in/2009/11/red5-server-installation-on-redhat-centos-linux/
http://cookingandcoding.com/2009/11/11/install-red5-xuggler-on-centos-5-x/

# wget -O /usr/local/src/jdk-6u20-linux-i586-rpm.bin wget http://cds.sun.com/is-bin/INTERSHOP.enfinity/WFS/CDS-CDS_Developer-Site/en_US/-/USD/VerifyItem-Start/jdk-6u20-linux-i586-rpm.bin?BundledLineItemUUID=IiiJ_hCuVakAAAEpBhRCxnzq&OrderID=5OSJ_hCuIasAAAEp.RNCxnzq&ProductID=guBIBe.oc_wAAAEnaDJHqPYe&FileName=/jdk-6u20-linux-i586-rpm.bin
# chmod 777 jdk-6u20-linux-i586-rpm.bin
# ./jdk-6u20-linux-i586-rpm.bin

Please enter “yes” or “no”.
Do you agree to the above license terms? [yes or no]
yes
Unpacking…
Checksumming…
Extracting…
UnZipSFX 5.50 of 17 February 2002, by Info-ZIP (Zip-Bugs@lists.wku.edu).
inflating: jdk-6u20-linux-i586.rpm
inflating: sun-javadb-common-10.5.3-0.2.i386.rpm
inflating: sun-javadb-core-10.5.3-0.2.i386.rpm
inflating: sun-javadb-client-10.5.3-0.2.i386.rpm
inflating: sun-javadb-demo-10.5.3-0.2.i386.rpm
inflating: sun-javadb-docs-10.5.3-0.2.i386.rpm
inflating: sun-javadb-javadoc-10.5.3-0.2.i386.rpm
Preparing… ########################################### [100%]
1:jdk ########################################### [100%]
Unpacking JAR files…
rt.jar…
jsse.jar…
charsets.jar…
tools.jar…
localedata.jar…
plugin.jar…
javaws.jar…
deploy.jar…
error: %post(jdk-1.6.0_20-fcs.i586) scriptlet failed, exit status 5
Installing JavaDB
Preparing… ########################################### [100%]
1:sun-javadb-common ########################################### [ 17%]
2:sun-javadb-core ########################################### [ 33%]
3:sun-javadb-client ########################################### [ 50%]
4:sun-javadb-demo ########################################### [ 67%]
5:sun-javadb-docs ########################################### [ 83%]
6:sun-javadb-javadoc ########################################### [100%]

Java(TM) SE Development Kit 6 successfully installed.

Product Registration is FREE and includes many benefits:
* Notification of new versions, patches, and updates
* Special offers on Sun products, services and training
* Access to early releases and documentation

Product and system data will be collected. If your configuration
supports a browser, the Sun Product Registration form for
the JDK will be presented. If you do not register, none of
this information will be saved. You may also register your
JDK later by opening the register.html file (located in
the JDK installation directory) in a browser.

For more information on what data Registration collects and
how it is managed and used, see:
http://java.sun.com/javase/registration/JDKRegistrationPrivacy.html

Press Enter to continue…..

Done.

# opera
ERROR: ld.so: object ‘libjvm.so’ from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object ‘libawt.so’ from LD_PRELOAD cannot be preloaded: ignored.
/usr/lib/opera/opera: error while loading shared libraries: libqt-mt.so.3: cannot open shared object file: No such file or directory

# yum install libqt-mt.so.3

http://schmidi2.blog.com/2008/05/17/opera-libjvmso-libawtso-cannot-be-preloaded-ignored-libjavaso-no-such-file-or-directory/

http://www.fedoraforum.org/forum/printthread.php?t=185372
http://translate.googleusercontent.com/translate_c?hl=en&sl=zh-CN&u=http://hi.baidu.com/dadazh/blog/category/unix%26%252347%253Blinux&prev=/search%3Fq%3DCentOS%2B%252B%2B%2522%2BMake%2Bsure%2Bany%2Brequired%2BX%2Bdevelopment%2Bpackages%2Bare%2Binstalled.%2522%26hl%3Den%26client%3Dfirefox-a%26hs%3DAfU%26rls%3Dorg.mozilla:en-US:official&rurl=translate.google.com&twu=1&usg=ALkJrhhYBLXntjTpj9tHTvymID9ULRiSmA

# yum install libX11-devel

1024X768

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

# x11vnc -usepw -display :5
28/06/2010 22:35:46 -usepw: found /root/.vnc/passwd
28/06/2010 22:35:46 x11vnc version: 0.9.10 lastmod: 2010-04-28 pid: 15437
28/06/2010 22:35:46 This x11vnc was built without X11 support (-rawfb only).

28/06/2010 22:35:46 ***************************************
28/06/2010 22:35:46 *** XOpenDisplay failed (:5)

*** x11vnc was unable to open the X DISPLAY: “:5”, it cannot continue.
*** There may be “Xlib:” error messages above with details about the failure.

Some tips and guidelines:

** An X server (the one you wish to view) must be running before x11vnc is
started: x11vnc does not start the X server. (however, see the -create
option if that is what you really want).

** You must use -display , -OR- set and export your $DISPLAY
environment variable to refer to the display of the desired X server.
– Usually the display is simply “:0” (in fact x11vnc uses this if you forget
to specify it), but in some multi-user situations it could be “:1”, “:2”,
or even “:137”. Ask your administrator or a guru if you are having
difficulty determining what your X DISPLAY is.

** Next, you need to have sufficient permissions (Xauthority)
to connect to the X DISPLAY. Here are some Tips:

– Often, you just need to run x11vnc as the user logged into the X session.
So make sure to be that user when you type x11vnc.
– Being root is usually not enough because the incorrect MIT-MAGIC-COOKIE
file may be accessed. The cookie file contains the secret key that
allows x11vnc to connect to the desired X DISPLAY.
– You can explicitly indicate which MIT-MAGIC-COOKIE file should be used
by the -auth option, e.g.:
x11vnc -auth /home/someuser/.Xauthority -display :0
x11vnc -auth /tmp/.gdmzndVlR -display :0
you must have read permission for the auth file.
See also ‘-auth guess’ and ‘-findauth’ discussed below.

** If NO ONE is logged into an X session yet, but there is a greeter login
program like “gdm”, “kdm”, “xdm”, or “dtlogin” running, you will need
to find and use the raw display manager MIT-MAGIC-COOKIE file.
Some examples for various display managers:

gdm: -auth /var/gdm/:0.Xauth
-auth /var/lib/gdm/:0.Xauth
kdm: -auth /var/lib/kdm/A:0-crWk72
-auth /var/run/xauth/A:0-crWk72
xdm: -auth /var/lib/xdm/authdir/authfiles/A:0-XQvaJk
dtlogin: -auth /var/dt/A:0-UgaaXa

Sometimes the command “ps wwwwaux | grep auth” can reveal the file location.

Starting with x11vnc 0.9.9 you can have it try to guess by using:

-auth guess

(see also the x11vnc -findauth option.)

Only root will have read permission for the file, and so x11vnc must be run
as root (or copy it). The random characters in the filenames will of course
change and the directory the cookie file resides in is system dependent.

See also: http://www.karlrunge.com/x11vnc/faq.html

HTMLDOC – html2image

htmldoc is a powerful simple-to-use tool/utility, which converts HTML to Postscript, PDF or indexed HTML output.

# convert /var/www/html/html2img/grid.ps -crop 280×300+40+9 -modulate 100 -geometry 530×480 /var/www/html/html2img/grid.png

here convert is ImageMagick tool for manipulating images..