Set up Amazon Web Services – Part 2
Dichotomy: Python 2 and 3
The Python world suffers from the contradictions between Python 2.x and 3. The latter is considered to be a paradise-like state, in which all teething troubles and glaring inconsistencies have been fixed, and cool new developments are taking place; except, hardly anyone uses Python 3 in production environments, and Amazon only offers 2.7.
In Python 2.x, programmers battle with a ludicrous sprawl of libraries and need to decide between incompatible products like urllib
and – I kid you not – urllib2
when fetching web data. If you want to run external software with 2.x, you would use check_output()
from the subprocess
module. In Python 3.x, however, the run()
method uses different parameters and check_output()
has apparently vanished.
Lambda Go
The Lambda function in Listing 2 takes the URL for the video file to be analyzed in the event
parameter dictionary under the movie_url
key. In a genuine production environment, no Python script can modify data in a fixed directory like data
and just hope that nobody else interferes. Because Amazon Lambda functions are called in parallel, they first need to create a temporary directory for the instance using Python's tempfile
module and then clean it up again when done.
To ensure that this happens if one of the functions throws an exception after a failure, the last line in a production script would be an exception handler; this was left out in the test version shown here. Line 10 of Listing 2 calls the mkdtemp()
method and uses the new directory to store the data determined in intermediate steps for the next stages in the script.
Line 16 stores the video file retrieved by a web request to a URL stored in the variable movie_file
; the last part of the path determines the local file name. The next step is for lines 19 to 21 to call the max-movement-lk.py
script in Listing 3 (this is a Python wrapper around the C program from Listing 1) and passes in the path to the video file located in the temporary directory.
Listing 3
max-movement-lk.py
On Site
If the analysis results in a number of JPEGs of format 0001.jpg
, 0002.jpg
, etc., the mk-montage.py
wrapper script from Listing 4 runs as of lines 24 and 25 in Listing 2. The script converts the seconds into the movie embedded in the file names to an HH::MM:SS
format and feeds the old file names and the formatted labels to ImageMagick's montage
tool:
Listing 4
mk-montage.py
montage.py -label 00:00:01 tmp/00001.jpg [...]
The program uses this to build a contact sheet in the montage.jpg
file, which ends up on Amazon's S3 storage system later, so that users can follow the link returned by the script later to download the content.
The Python script in Listing 5 acts as a wrapper around the montage
binary, which is accompanied by a bunch of shared libraries in the lib
directory so that the dynamically linked binary will run in the container. The LD_LIBRARY_PATH
environment variable sets the search path for shared libs on the non-standard directory so that the binary actually finds them at run time.
Listing 5
montage.py
Figure 4 shows the shared libs collected by Listing 6. Apparently, the AI program for motion analysis linked with OpenCV drags a trunk full of dependent libraries around with it. Video analysis requires sophisticated decompression technology if you want access to the raw frame data.
Listing 6
ldd-ls.py
« Previous 1 2 3 Next »
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Linux Servers Targeted by Akira Ransomware
A group of bad actors who have already extorted $42 million have their sights set on the Linux platform.
-
TUXEDO Computers Unveils Linux Laptop Featuring AMD Ryzen CPU
This latest release is the first laptop to include the new CPU from Ryzen and Linux preinstalled.
-
XZ Gets the All-Clear
The back door xz vulnerability has been officially reverted for Fedora 40 and versions 38 and 39 were never affected.
-
Canonical Collaborates with Qualcomm on New Venture
This new joint effort is geared toward bringing Ubuntu and Ubuntu Core to Qualcomm-powered devices.
-
Kodi 21.0 Open-Source Entertainment Hub Released
After a year of development, the award-winning Kodi cross-platform, media center software is now available with many new additions and improvements.
-
Linux Usage Increases in Two Key Areas
If market share is your thing, you'll be happy to know that Linux is on the rise in two areas that, if they keep climbing, could have serious meaning for Linux's future.
-
Vulnerability Discovered in xz Libraries
An urgent alert for Fedora 40 has been posted and users should pay attention.
-
Canonical Bumps LTS Support to 12 years
If you're worried that your Ubuntu LTS release won't be supported long enough to last, Canonical has a surprise for you in the form of 12 years of security coverage.
-
Fedora 40 Beta Released Soon
With the official release of Fedora 40 coming in April, it's almost time to download the beta and see what's new.
-
New Pentesting Distribution to Compete with Kali Linux
SnoopGod is now available for your testing needs