Personal Data Manager
khal and khard
In addition to vdirsyncer, two other tools, khard [8] for address books and khal [9] for calendars, offer a simple solution for this tutorial. Khard is the most essential. Khal can be used interactively: Figure 3 displays the same calendar shown in Figure 2 read from the local copy previously created by vdirsyncer. Both khard and khal can be installed from the standard repositories of major Linux distributions.
Khal's configuration can be as simple as Listing 6: The calendars
section lists all the desired calendars, while the default
section sets the options, most with self-explanatory names, that are common to all calendars: default calendar, number of days to display (timedelta
), highlighting, and date formatting.
Listing 6
Configuring khal
[calendars] [[work]] path = ~/.pim/work type = calendar color = dark green [[personal]] path = ~/.pim/personal type = calendar color = yellow [default] default_calendar = work highlight_event_days = true timedelta = 15d #Dates formatting [locale] local_timezone= Europe/Rome firstweekday = 0 timeformat = %H:%M dateformat = %Y/%m/%d longdateformat = %Y/%m/%d datetimeformat = %Y/%m/%d %H:%M longdatetimeformat = %Y/%m/%d %H:%M
Khard's configuration (Listing 7) can be almost as simple, if you only use it for importing bulk contacts or as an address grabber for console email clients like Mutt (more on this later).
Listing 7
Configuring khard
[addressbooks] [[All]] path = ~/.pim/contacts [general] debug = no default_action = list editor = gedit merge_editor = /usr/share/doc/khard/examples/sdiff/sdiff_khard_wrapper.sh [contact table] display = first_name group_by_addressbook = no sort = last_name
The editor
and merge_editor
options need some additional explanation. If these options do not point to existing programs, khard will not start. In Listing 7, the merge_editor
used to handle conflicts is the one included in the khard package.
If you launch khard without options, it will just list all the contacts it found. In my case, the contacts shown in Listing 8 are the same ones that I had created in Nextcloud.
Listing 8
List of Contacts
#> khard Address book: All Index Name Phone E-Mail UID 1 Bruce Wayne HOME,VOICE: 555-5555 HOME: batman@example.com b 2 Superman HOME,VOICE: HOME: 3
You can add new contacts on the command line. For the default address book, just type
#> khard new
and answer khard's questions. If you need to work on a non-default address book, specify it with the -a
switch. It is also possible to configure Mutt to add or fetch addresses from khard, with the following two instructions that are explained in detail in the khard documentation [10]:
macro pager,index a "<pipe-message>khard add-email<return>" "add the sender address to khard" set query_command= "khard email --parsable %s"
Khard and khal are very simple, very quick ways to read or write personal data from a Linux terminal. Their real power, however, is their support for automatic import and export of data to or from Nextcloud or any other CalDAV/CardDAV server. To load two new events into my Nextcloud work calendar (Figure 4) I used the following commands, and then ran vdirsyncer again:
khal -c khal-config-local.conf new -a work '2020/03/24 09:00' '2020/03/24 11:30' 'Engineers meeting' khal -c khal-config-local.conf new -a work '2020/03/25 09:00' '2020/03/25 18:00' 'Project meeting'
With khal, you can add hundreds of events to multiple calendars with a shell script that reads the events to be added from a plain text file. The plain text file may be a spreadsheet in CSV format, a database dump, or anything else, as long as it stores one event per line in a format similar to:
Engineers meeting|work|2020/03/24 09:00|2020/03/24 11:30
A script would need to read that file one line at a time, assigning each pipe-separated field to a variable (e.g., in the example above, $EVENT
, $CALENDAR
, $START
, and $END
) and then launch khal as follows, once per line:
khal -c $CONFIG_FILE new -a $CALENDAR $START $END $EVENT
In a real script, the above command should be modified to handle spaces inside each variable. But once the input file has been read, one call to vdirsyncer will copy all those events to the Nextcloud calendar.
Importing addresses with khard works in the same way, with one important difference: With khard, you must write all the contact data in a template generated by khard with the following command:
khard -c khard.conf show --format=yaml > template.yaml
Once you have the template, you can replace the value of all parameters with the right ones for your contact inside the file. Listing 9 shows the content of the template.yaml
file, when filled with Wonder Woman's contact information; Figure 5 shows the corresponding contact in the Nextcloud address book, after running the following khard
command:
khard -c khard.conf new -i template.yaml
Listing 9
template.yaml
Prefix : Princess First name : Diana Additional : Last name : Themishira Suffix : Nickname : Anniversary : Birthday : Organisation : League of Justice Title : Role : Phone : HOME,VOICE : 555-5556 Email : HOME : wonderwoman@example.com Address : WORK: Box : Extended : Street : Code : City : Region : Mount Olympus Country : Planet Earth Categories : Webpage : https://en.wikipedia.org/wiki/Wonder_Woman Private : Jabber : Skype : Twitter : wonderwoman_official Note :
Just like with khal, you can use the khard
command above inside a loop, which fills the template with new values read from a plain text file at every iteration and calls vdirsyncer when it has finished.
Backups and Post-Processing
As a side benefit of vdirsyncer, you can maintain a full backup of all your CalDAV/CardDAV data, regardless of what you do with it. However, that is not the only way to export or back up that data, and sometimes it is not even the best one. If you want to back up all of the personal data of all your Nextcloud installation's users, for example, you can run the Bash script, calcardbackup
[11], on the server as follows:
sudo -u www-data /PATH/TO/calcardbackup $NEXTCLOUD_ROOT_FOLDER
The result will be a tar archive containing several .ics
or .vcf
files with names like USERNAME-CALENDARNAME
, with each file containing one whole calendar or address book. The calcardbackup
website also explains how to compress or encrypt backups or just back up the data of selected users.
Personally, I use calcardbackup
any time I need to export all my calendar events, in a simpler CSV format, which is much better suited for further processing. To do that, I wrote a very quick and dirty script called ics2csv.pl
(Listing 10). Listing 11 shows how to run the script, and its resulting output.
Listing 10
ics2csv.pl
01 #! /usr/bin/perl 02 use strict; 03 use POSIX 'strftime'; 04 05 my $TODAY = strftime '%Y%m%d', localtime; 06 my ($CALENDAR, $SUMMARY, $START, $STARTDAY, $END, $ENDDAY, $STARTHOUR, $ENDHOUR); 07 my @APPOINTMENTS; 08 09 print "## All events starting from $TODAY\n\n"; 10 11 while (<>) { 12 next unless ($_ =~ m/^(X-WR-CALNAME|SUMMARY|DTSTART|DTEND|END:VEVENT)/); 13 14 s/;VALUE=DATE//; 15 s/;TZID=Europe\/Rome//; 16 17 if ($_ =~ m/^X-WR-CALNAME:(.*)\r\n/) { $CALENDAR = $1; $CALENDAR = lc($CALENDAR)} 18 19 if ($_ =~ m/^SUMMARY:(.*)\r\n/){ $SUMMARY = $1; } 20 21 if ($_ =~ m/^DTSTART:(\w*)/) { 22 $START = $1; 23 $STARTDAY = substr($START, 0,8); 24 $STARTHOUR = substr($START, 9); 25 substr($STARTDAY,4,0) = '-'; 26 substr($STARTDAY,7,0) = '-'; 27 if (length($STARTHOUR) > 0) { 28 substr($STARTHOUR,2,0) = ':'; 29 substr($STARTHOUR,-2) = ''; 30 } 31 } 32 33 if ($_ =~ m/^DTEND:(\w*)/) { 34 $END = $1; 35 $ENDDAY = substr($END, 0,8); 36 $ENDHOUR = substr($END, 9); 37 substr($ENDDAY,4,0) = '-'; 38 substr($ENDDAY,7,0) = '-'; 39 if (length($ENDHOUR) > 0) { 40 substr($ENDHOUR,2,0) = ':'; 41 substr($ENDHOUR,-2) = ''; 42 } 43 } 44 45 if ($_ =~ m/^END:VEVENT/) { 46 47 push (@APPOINTMENTS,sprintf("%10.10s %5.5s | %-10.10s | %-35.35s | %10.10s %5.5s\n", $STARTDAY,$STARTHOUR, $CALENDAR, $SUMMARY,$ENDDAY, $ENDHOUR)); 48 } 49 } 50 51 for my $APP (sort @APPOINTMENTS) { 52 my $CURRENTDAY = substr($APP,0,10); 53 $CURRENTDAY=~ s/-//g; 54 print "$APP\n" if ($CURRENTDAY >= $TODAY); 55 }
Listing 11
Running ics2csv.pl
#> ics2csv.pl calcardbackup-2030-03-01/marco-work.ics > calendar.csv #> cat calendar.csv ## All events starting from 20200301 2020-03-28 | work | FOSSMeet talk | 2020-03-29 2020-03-30 | work | Linuxmag: caldav servers | 2020-03-31
Listing 10 shows the power of storing your personal data in any CalDAV/CardDAV server, be it Nextcloud or anything else. Once you have done that, not only can you access the data anywhere on any device, the data also becomes very easy to reuse. It took me less than 15 minutes, testing included, to put together the whole thing (Listing 10), including copying and pasting.
Lines 1 to 10 of Listing 10 call the Perl libraries that perform several variable checks or provide time formatting functions. They then define all the variables I need and print the header line. Lines 11 and 12 scan the .ics
file passed as an argument, skipping all lines except those that start with any of the strings in line 12.
Line 17 spots the variable containing the calendar name and saves its lowercase version into $CALENDAR
. Line 19 does the same job for the event summary.
Lines 21 to 31 recognize and reformat the current event's start date, which is DTSTART
in .ics
files. Lines 33 to 43 do the same for the end date.
Whenever it finds the string END:VEVENT
(line 45), the script knows that the description of an event has finished, and it appends a string with the calendar name, start time, event description, and end time (separated by pipes) to an array called @APPOINTMENTS
.
After the whole .ics
file has been scanned, all events are written to standard output, in chronological order (lines 51 to 55).
Radicale
If Nextcloud doesn't meet your needs, another option is the lightweight CalDAV/CardDAV server called Radicale [12]. A Python 3 package, Radicale hosts multiple calendars and contacts for multiple users, with several options to restrict and log accesses. It does not support all the CalDav/CardDAV specifications' features, but it is adequate for small scale usage. Above all, Radicale is very simple to install and configure. Besides Python 3, its only dependencies are the htpasswd
utility to encrypt user passwords and the Python 3 libraries to decrypt them.
Radicale's website explains step by step how to install Radicale with the pip package manager and run it for one single user. However, I have found that on Ubuntu and other distributions, a Radicale multi-user installation is as simple as typing the following:
#> sudo apt-get install radicale apache2-utils python3-bcrypt python3-passlib #> systemctl start radicale
This makes it much easier to keep current afterwards.
The apache2-utils package contains the htpasswd
program, and the other two packages contain the required Python libraries. Radicale's default configuration is enough to run and access Radicale only on your local machine. To make it accessible from other computers or from the Internet, you need to "bind" Radicale to other addresses. To learn how to do this simple process, see Radicale's documentation [12]. I strongly suggest, however, that you play with a local-only installation before going live. For that, the only critical settings in /etc/radicale/config
define the location and encryption method of usernames and passwords as follows:
[auth] type = htpasswd htpasswd_filename = /et/radicale/users htpasswd_encryption = bcrypt
To create a Radicale user and store the user's encrypted passwords in htpasswd_filename
, run the following command for each user and follow its instructions:
sudo htpasswd -B /etc/radicale/users marco
Now, you can restart Radicale, point your browser to http://localhost:5232/, log in, and start creating calendars and address books in the interface shown in Figure 6. Of course, since Radicale is only a server, you will need clients like khal, khard, or Thunderbird to actually read and write those databases. To import personal data from other servers, just use vdirsyncer, and it will create automatically all the desired calendars and address books. When I configured vdirsyncer as shown in Listing 12, it uploaded all the data previously copied on my computer from the Nextcloud server to my Radicale server (see Figure 7).
Listing 12
Syncing Radicale to Nextcloud with vdirsyncer
[general] status_path = "~/.vdirsyncer/status/" [pair cal_local_to_radicale] a = "my_cal_local" b = "my_cal_radicale" collections = ["from a", "from b"] [storage my_cal_local] type = "filesystem" path = "~/.pim/" fileext = ".ics" [storage my_cal_radicale] type = "caldav" url = "http://localhost:5232" username = "my_radicale_user_name" password = "my_radicale_password"
« Previous 1 2 3 Next »
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
So Long Neofetch and Thanks for the Info
Today is a day that every Linux user who enjoys bragging about their system(s) will mourn, as Neofetch has come to an end.
-
Ubuntu 24.04 Comes with a “Flaw"
If you're thinking you might want to upgrade from your current Ubuntu release to the latest, there's something you might want to consider before doing so.
-
Canonical Releases Ubuntu 24.04
After a brief pause because of the XZ vulnerability, Ubuntu 24.04 is now available for install.
-
Linux Servers Targeted by Akira Ransomware
A group of bad actors who have already extorted $42 million have their sights set on the Linux platform.
-
TUXEDO Computers Unveils Linux Laptop Featuring AMD Ryzen CPU
This latest release is the first laptop to include the new CPU from Ryzen and Linux preinstalled.
-
XZ Gets the All-Clear
The back door xz vulnerability has been officially reverted for Fedora 40 and versions 38 and 39 were never affected.
-
Canonical Collaborates with Qualcomm on New Venture
This new joint effort is geared toward bringing Ubuntu and Ubuntu Core to Qualcomm-powered devices.
-
Kodi 21.0 Open-Source Entertainment Hub Released
After a year of development, the award-winning Kodi cross-platform, media center software is now available with many new additions and improvements.
-
Linux Usage Increases in Two Key Areas
If market share is your thing, you'll be happy to know that Linux is on the rise in two areas that, if they keep climbing, could have serious meaning for Linux's future.
-
Vulnerability Discovered in xz Libraries
An urgent alert for Fedora 40 has been posted and users should pay attention.