Building a Web Spider with Ruby
Spider on the Web
Ruby is a very elegant language,and it’s harmonious – the parts work together effectively. Ruby also significantly reduces a developer’s burden. We’ll show you how to use Ruby to build a quick and simple web spider application.
Ruby is a scripting language developed by Yukihiro Matsumoto and released under the GPL. The Ruby language has an excellent set of string manipulation and networking libraries, making it a great choice for writing web spiders. If you are not familiar with web spiders, they are programs designed to automatically traverse the web. Search engines use web spiders to add web pages to their index; companies like Netcraft use spiders to get statistics on web servers. You can use a web spider to find information automatically from almost any website; in this article, we’ll discuss how to use Ruby to retrieve information from LiveJournal, a popular weblog provider. You can extend these techniques to virtually every website that provides public information.
Read full article as PDF:
Ruby_Web_Spiders.pdf (165.09 kB)Tag Cloud
News
-
KDE Announces Software Compilation 4.13
New release comes with better semantic search and improvements to Kontact.
-
Coverity: Open Source Code has Fewer Defects
Annual code quality report shows FOSS is more secure at all project size levels.
-
An Even Smaller Raspberry Pi
The Raspberry Pi Foundation has announced an even smaller version of the tiny computer that will fit into a DIMM slot.
-
Freaky Privilege Escalation Attack
A new class of problems lets a malicious app pre-configure an invisible privilege update.
-
Facebook Rolls Out New PHP-like Programming Language
New Hack language adds static typing and other conveniences.
-
Fedora Announces Innovative Crytography Policy System
New crypto policy system will offer easier configuration and more uniform security.
-
Shuttleworth Calls for Declarative Firmware
Ubuntu founder denounces insecurity in proprietary, close-source software blobs.
-
New TLS Attack Takes the S out of HTTPS
Vulnerability affects many Linux web servers
-
Munich Adopts Kolab
The Bavarian capital shuns Microsoft, Google, and other alternatives to implement an open source groupware solution.
-
Canonical Announces Ubuntu Smartphones
Phone vendor partnerships bring Mark Shuttleworth's dream of Ubuntu on a phone a step closer to reality.