Building a Web Spider with Ruby
Spider on the Web
Ruby is a very elegant language,and it’s harmonious – the parts work together effectively. Ruby also significantly reduces a developer’s burden. We’ll show you how to use Ruby to build a quick and simple web spider application.
Ruby is a scripting language developed by Yukihiro Matsumoto and released under the GPL. The Ruby language has an excellent set of string manipulation and networking libraries, making it a great choice for writing web spiders. If you are not familiar with web spiders, they are programs designed to automatically traverse the web. Search engines use web spiders to add web pages to their index; companies like Netcraft use spiders to get statistics on web servers. You can use a web spider to find information automatically from almost any website; in this article, we’ll discuss how to use Ruby to retrieve information from LiveJournal, a popular weblog provider. You can extend these techniques to virtually every website that provides public information.
Read full article as PDF:
Ruby_Web_Spiders.pdf (165.09 kB)Tag Cloud
News
-
An Even Smaller Raspberry Pi
The Raspberry Pi Foundation has announced an even smaller version of the tiny computer that will fit into a DIMM slot.
-
Newly Discovered Android Vulnerability Causes Privilege Escalation
A new class of problems lets a malicious app pre-configure an invisible privilege update.
-
Facebook Rolls Out New PHP-like Programming Language
New Hack language adds static typing and other conveniences.
-
Fedora Announces Innovative Crytography Policy System
New crypto policy system will offer easier configuration and more uniform security.
-
Shuttleworth Calls for Declarative Firmware
Ubuntu founder denounces insecurity in proprietary, close-source software blobs.
-
New TLS Attack Takes the S out of HTTPS
Vulnerability affects many Linux web servers
-
Munich Adopts Kolab
The Bavarian capital shuns Microsoft, Google, and other alternatives to implement an open source groupware solution.
-
Canonical Announces Ubuntu Smartphones
Phone vendor partnerships bring Mark Shuttleworth's dream of Ubuntu on a phone a step closer to reality.
-
Piviti Seeks Funding
Donors will get to vote on new features for the free video editor.
-
Debian Tech Committee Votes for systemd
Debian project puts init out to pasture and says no to Ubuntu's Upstart.