Finding and retrieving Google Drive files with Go
Seal of Approval Still Missing
Google then asks the user logged in with their Google account whether they want to grant the app read access to their Google Drive data. Since the Go program is a homemade app and does not yet bear the Google seal of approval, the dialog in Figure 7 warns the user against granting access. Brave readers of this column will want to press Advanced and grant the Go program read access to their Google Drive anyway on the following page.
After confirming again, the Google OAuth 2 flow finally outputs a hex code (Figure 8), which you need to copy to the standard input of the Go program that has been waiting at the command line. When you type Enter
, the program devours the code and continues to run. It contacts the Google server with the code and receives an access and refresh token from the server. Listing 1 then bundles these credentials into a local JSON file, which it stores as token.json
in the same directory.
Now that it has persistent credentials, Listing 1 no longer sends the user through the OAuth 2 flowwhen next called, but can use the access token in the JSON file to read the user's data on Google Drive. It is important to protect this JSON file against unauthorized access: Anyone in possession of the token can gain access to your Google Drive data. However, write access is not possible, because the scope was previously defined as read-only during the OAuth 2 flow.
Hop, Skip, and Google
Listing 2 implements the oauth2Client
function to let the program collect and manage the token when run for the first time. When done, it returns an HTTP client to the main program, which handles user authentication under the hood when communicating with the Google Drive web server.
Listing 2
oauth2.go
01 package main 02 03 import ( 04 "encoding/json" 05 "fmt" 06 "golang.org/x/net/context" 07 "golang.org/x/oauth2" 08 "log" 09 "net/http" 10 "os" 11 ) 12 13 func oauth2Client(config *oauth2.Config) *http.Client { 14 tokFile := "token.json" 15 tok, err := readCachedToken(tokFile) 16 if err != nil { 17 tok = fetchAccessToken(config) 18 cacheToken(tokFile, tok) 19 } 20 return config.Client(context.Background(), tok) 21 } 22 23 func fetchAccessToken(config *oauth2.Config) *oauth2.Token { 24 url := config.AuthCodeURL("state-token", oauth2.AccessTypeOffline) 25 fmt.Printf("Point browser to %v, follow the flow and then paste "+ 26 "the code here.\n", url) 27 28 var authCode string 29 if _, err := fmt.Scan(&authCode); err != nil { 30 log.Fatalf("Error reading auth code %v", err) 31 } 32 33 tok, err := config.Exchange(context.TODO(), authCode) 34 if err != nil { 35 log.Fatalf("Error getting access token: %v", err) 36 } 37 return tok 38 } 39 40 func readCachedToken(file string) (*oauth2.Token, error) { 41 f, err := os.Open(file) 42 if err != nil { 43 return nil, err 44 } 45 defer f.Close() 46 tok := &oauth2.Token{} 47 err = json.NewDecoder(f).Decode(tok) 48 return tok, err 49 } 50 51 func cacheToken(path string, token *oauth2.Token) { 52 fmt.Printf("Saving credential file to: %s\n", path) 53 f, err := os.OpenFile(path, os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0600) 54 if err != nil { 55 log.Fatalf("Can't write token to %s: %v", path, err) 56 } 57 defer f.Close() 58 json.NewEncoder(f).Encode(token) 59 }
When the program is first called, there is no token in the token.json
file yet, so calling readCachedToken()
in line 15 returns an error. This is then remedied by calling the fetchAccessToken()
function in line 17. Starting in line 23, the code prints out a Google URL, which you need to paste into your web browser; from there, you then go through the OAuth 2 flow.
At the end, the flow shows a hex code that the user then copies into the main program waiting at the command line. Line 29 grabs the code from the standard input; line 33 exchanges it on the Google server for an access token, which line 18 stores in JSON format on the local filesystem. On subsequent invocations of the program, the code instructs oauth2Client()
to retrieve the token from the local file without the previously necessary rigamarole. No more extra Google hops needed at this point.
The json package from the Go core collection makes reading and writing token data a breeze. Juggling the tokens to gain access to a protected resource isn't as convenient; someone might want to sit down and provide a standardized token handling package on GitHub for everyone else to use.
The pickNGet()
function in Listing 3 sends the search query to Google Drive. The service stores files in a freely definable hierarchy of folders. However, users often simply search for file content or names, and the search engine giant returns a unique ID for matching files.
Listing 3
pick.go
01 package main 02 03 import ( 04 "bufio" 05 "fmt" 06 pb "github.com/schollz/progressbar/v3" 07 "google.golang.org/api/drive/v3" 08 "log" 09 "os" 10 "strings" 11 ) 12 13 func pickNGet(srv *drive.Service, query string) error { 14 q := fmt.Sprintf("name contains '%s'", query) 15 r, err := srv.Files.List().Q(q).PageSize(100). 16 Fields("nextPageToken, files(id, name, size)").Do() 17 if err != nil { 18 log.Fatalf("Error retrieving files: %v", err) 19 } 20 21 if len(r.Files) == 0 { 22 fmt.Println("No files found.") 23 return nil 24 } 25 26 reader := bufio.NewReader(os.Stdin) 27 28 for _, file := range r.Files { 29 fmt.Printf("Download %s (y/[n])? ", file.Name) 30 text, _ := reader.ReadString('\n') 31 if !strings.Contains(text, "y") { 32 continue 33 } 34 bar := pb.DefaultBytes(file.Size, "downloading") 35 36 fmt.Printf("Downloading %s (%d) ...\n", file.Name, file.Size) 37 dwn, err := srv.Files.Get(file.Id).Download() 38 if err != nil { 39 log.Fatal("Unable to get download link %v", err) 40 } 41 defer dwn.Body.Close() 42 43 reader := bufio.NewReader(dwn.Body) 44 err = download(reader, file.Name, bar) 45 if err != nil { 46 log.Fatal("Download of %s failed: %v", file.Name, err) 47 } 48 } 49 50 return nil 51 }
My file names are usually unique, which is why line 14 in Listing 3 uses the search query name contains x
. If you prefer to let Google search for text chunks in the content, just replace the string in line 14 with fullText contains x
. Further search queries are explained in the API document [4].
Machine-Generated SDK
It turns out that the Go SDK for the Google API provided by Google is simply a machine-generated wrapper around the web API.
The API endpoint for listing files is addressed by srv.Files.List()
(line 15). The concatenated calls to Q(q).PageSize(100)
append the search query and set the maximum number of matches delivered for each request to 100. If so desired, the user can pick up the next batch of matches on the next call, but Listing 3 does not do this, because a command-line client would be unsuitable for processing more than 100 matches anyway.
The concatenated Fields()
function limits the fields returned per match to the unique document ID, the file name, and the size of the file in bytes. pickNGet()
can therefore offer the user a compact list for selection. Pressing Y starts the download.
The for
loop starting in line 28 iterates over all the matches in the r.Files
array and prompts the user on os.Stdin
at each pass to press Y if they want to download the current match to their local drive. To allow this to happen, the ReadString()
function in line 30 waits for keyboard input from the user, which is sent after the Enter key gets pressed. If the user did not type y
(e.g. by accepting the default "n"
), line 32 uses continue
to go to the next round, asking the user what they want to do with the next match.
If the user wants to download a matching file, line 34 sets the progress bar to zero percent and the maximum length to the size of the file. The Google Drive call srv.Files.Get()
in line 37 selects the desired file by referencing its ID and calls Download()
to initiate the download. In this case, the Google Drive server sends back a download URL, which the client docks onto, and the download process starts via HTTPS.
Line 43 defines a buffered reader from the standard bufio package for the incoming data stream. In line 44, the program passes the reader to the download()
function from Listing 4, together with a reference to the progress bar bar
and the file name to store it under later locally.
Listing 4
download.go
01 package main 02 03 import ( 04 "bufio" 05 pb "github.com/schollz/progressbar/v3" 06 "io" 07 "os" 08 ) 09 10 func download(r io.Reader, lpath string, bar *pb.ProgressBar) error { 11 outf, err := os.OpenFile(lpath, os.O_WRONLY|os.O_CREATE, 0644) 12 if err != nil { 13 return err 14 } 15 writer := bufio.NewWriter(outf) 16 defer outf.Close() 17 18 total := 0 19 data := make([]byte, 1024*1024) 20 21 for { 22 count, rerr := r.Read(data) 23 if rerr != io.EOF && rerr != nil { 24 return err 25 } 26 total += count 27 bar.Add(count) 28 realdata = data[:count] 29 30 _, werr := writer.Write(realdata) 31 if werr != nil { 32 return werr 33 } 34 35 if rerr == io.EOF { 36 break 37 } 38 } 39 writer.Flush() 40 return nil 41 }
« Previous 1 2 3 4 Next »
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
So Long Neofetch and Thanks for the Info
Today is a day that every Linux user who enjoys bragging about their system(s) will mourn, as Neofetch has come to an end.
-
Ubuntu 24.04 Comes with a “Flaw"
If you're thinking you might want to upgrade from your current Ubuntu release to the latest, there's something you might want to consider before doing so.
-
Canonical Releases Ubuntu 24.04
After a brief pause because of the XZ vulnerability, Ubuntu 24.04 is now available for install.
-
Linux Servers Targeted by Akira Ransomware
A group of bad actors who have already extorted $42 million have their sights set on the Linux platform.
-
TUXEDO Computers Unveils Linux Laptop Featuring AMD Ryzen CPU
This latest release is the first laptop to include the new CPU from Ryzen and Linux preinstalled.
-
XZ Gets the All-Clear
The back door xz vulnerability has been officially reverted for Fedora 40 and versions 38 and 39 were never affected.
-
Canonical Collaborates with Qualcomm on New Venture
This new joint effort is geared toward bringing Ubuntu and Ubuntu Core to Qualcomm-powered devices.
-
Kodi 21.0 Open-Source Entertainment Hub Released
After a year of development, the award-winning Kodi cross-platform, media center software is now available with many new additions and improvements.
-
Linux Usage Increases in Two Key Areas
If market share is your thing, you'll be happy to know that Linux is on the rise in two areas that, if they keep climbing, could have serious meaning for Linux's future.
-
Vulnerability Discovered in xz Libraries
An urgent alert for Fedora 40 has been posted and users should pay attention.