Building a Web Spider with Ruby
Spider on the Web
Ruby is a very elegant language,and it’s harmonious – the parts work together effectively. Ruby also significantly reduces a developer’s burden. We’ll show you how to use Ruby to build a quick and simple web spider application.
Ruby is a scripting language developed by Yukihiro Matsumoto and released under the GPL. The Ruby language has an excellent set of string manipulation and networking libraries, making it a great choice for writing web spiders. If you are not familiar with web spiders, they are programs designed to automatically traverse the web. Search engines use web spiders to add web pages to their index; companies like Netcraft use spiders to get statistics on web servers. You can use a web spider to find information automatically from almost any website; in this article, we’ll discuss how to use Ruby to retrieve information from LiveJournal, a popular weblog provider. You can extend these techniques to virtually every website that provides public information.
Read full article as PDF:
Ruby_Web_Spiders.pdf (165.09 kB)Tag Cloud
News
-
Munich Adopts Kolab
The Bavarian capital shuns Microsoft, Google, and other alternatives to implement an open-source groupware solution.
-
Canonical Announces Ubuntu Smartphones
Phone vendor partnerships bring Mark Shuttleworth's dream of Ubuntu on a phone a step closer to reality.
-
Piviti Seeks Funding
Donors will get to vote on new features for the free video editor.
-
Debian Tech Committee Votes for systemd
Debian project puts init out to pasture and says no to Ubuntu's Upstart.
-
The Mask: Scary New Face of Internet Intrusion
Ultra-sophisticated attack tool might have originated from a state-sponsored intelligence service.
-
Epoch 1.0 Released
New alternative for init comes with a small footprint and minimal configuration.
-
Wayland 1.4 Challenges X11
X marks the target for the next-generation windowing system.
-
Red Hat Adopts CentOS
Super-clone CentOS Linux gets beamed up to the mother ship.
-
W3C Promotes Media Source Extensions
HTML technology will enable new video editing and playback options.
-
Valve Announces SteamOS Beta
New Linux distro is optimzed for gaming.