Ssscrape 1.0 Collects Dynamic Web Data
The Ssscrape tool screen-scrapes data from RSS and Atom feeds, blogs and podcasts. The open source software is now available in version 1.0.
Ssscrape tracks feeds and other collections for similar elements on updates, and downloads and cleans content by converting HTML to plain text. The database used is MySQL. The tool can also gather statistics about feed activities and report errors. A scheduler takes care of the periodic checks and a monitor displays the running activities.
Known as a Web crawler, a program that scrapes together information off the Web, Ssscrape is short for Syndicated and Semi-Structured Content Retrieval and Processing Environment. The Web scraper is written in Python with Twisted used for network programming and the not always standards-based Beautiful Soup used for parsing HTML/XML content.
Ssscrape was developed in the Information and Language Processing Systems (ILPS) department of the University of Amsterdam and is under LGPLv3 licensing. Ssscrape 1.0 requires Python 2.4 and is available for download as a tarball from the project page.
Issue 14: Raspberry Pi Handbook/Special Editions
Tag Cloud
News
-
SCO Rises from the Swamp
Longtime litigator revives an ancient suit against IBM alleging Linux infringes on Unix copyrights.
-
UberStudent Project Releases UberStudent 3.0
Specialty distro keeps the focus on advanced learning.
-
openSUSE Conference Approaches
The openSUSE Conference will be held July 18-22, 2013, at the Olympic Museum in Thessaloniki, Greece.
-
Drupal.org Hacked
Security breached at home sites of the CMS project.
-
Oracle Takes Action on Java Security
Lead Java developer vows policy changes and more attention to fixing problems.
-
Google and NASA Partner in Quantum Computing Project
Vendor D-Wave scores big with a sale to NASA's Quantum Intelligence Lab.
-
Mageia Project Announces Mageia 3 Linux
Many package updates and Steam integration highlight the latest from the Mandriva-based community Linux.
-
FSF Outs the World Wide Web Consortium over DRM Proposal
Richard Stallman calls for the W3C to remain independent of vendor interests.
-
Debian 7.0 Debuts
The new release supports nine architectures, 73 human languages, and zero non-Free components.
-
Alpha Version of Fedora 19 Released
Fedora developers release the first alpha version of Fedora 19, known as Schrödinger’s Cat, for general testing. The final release is expected in July 2013.