Automatic Files Upload to Github on Mac

with shell scripts and Launchd

I have been planning to do this for some time. On my computer desktop there is a folder named ‘notes’ where I keep my, well, notes. I use Git to version control it (because why not), so I can access the notes from both my work and personal computer.

It is convenient–no additional software, a terminal and a few commands are all I need–but not enough. I need to issue the commands to upload the files everyday. Even if the commands are just three lines. Even if it takes only 10 seconds. More importantly, if I forget to do it, the notes would not get updated (duh), which sometimes can be annoying.

So the idea is simple. Write a script that contains the necessary commands for uploading(a.k.a. the three lines), and tell the computer to run it at a schedule. For the first part we need shell script. For the second part we need launchd.

The Shell Script

A shell script is basically a file that you can execute in the shell.

It can be as simple as a glob of shell commands lumped together (example below). Or it can be as powerful as system protection and data tracking.

The language used in shell scripts are called scripting language. You might have heard of this term along with dynamic high-level languages like Python or Ruby. These languages are executed line by line–you see the immediate result of the execution of each line. On the other end of the spectrum would be low-level languages, the ones that need to be compiled first before running, like C.

In our case, we will use Bash syntax:

https://gist.github.com/jenny-codes/8862a38a72df5ed403421b251a78315e

And you thought writing a shell script is hard.

The first line of the script I unabashedly presented above is

#!/bin/bash

It tells the system that this script is to be executed by bash. Similarly, you can replace the line with #!/usr/bin/ruby to execute the script with Ruby, or any other scripting language to your taste.

The second line translates as “if there is an argument passed in, assign the value to variable msg, or else assign ‘regular update’ to msg.” The variable msg, as we can see at line five, will be the commit message of this upload.

Put this script anywhere you like. Call it whatever you want. I put my file directly at usr/local/bin, and named it git_push_all. Note that some will add a suffix .sh to the script filename, but this is not obligatory.

The rest really needs no further explaining, but

Make sure you script is executable

is very important. Type this line if you haven’t already:

chmod +x filename.sh

The command chmod, short for ‘change mode’, changes permissions on the file filename.sh. Argument +x grants execution permission to all three categories of users (the file owner, the group where the file belongs, others). We can use ls -l in the directory to double check. See this good RedHat article for detailed instructions and explanations.

Test the Script

You can test it by navigating to the directory where you put the file and type ./git_push_all. Or if you put it in one of your execution paths (declared in $PATH environment variable) like I did, you can use git_push_all anywhere you wish. Of course, it should be used in a git directory or it would be useless. You should see the usual output messages printed on the console.

Automate Our Script with Launchd

Now that we have our script ready, our next step is put it on our machine’s schedule so it can run by itself. We will use MacOS’ built-in Launchd for our task. Launchd is a service management framework on the operating system level used by MacOS. Its job is to manage various daemons (a fancy term for ‘background programs’) in the system.

Launchd vs Cron

When it comes to job scheduler, the term ‘cron’ usually comes along. Perplexing enough, MacOS provides a service named crontab as well, which seems to have the same functionality as Launchd. It is worth investing some time to figure out their relationship. Let us look at the man definition first:

[CRON]

NAME
 cron — daemon to execute scheduled commands (Vixie Cron)
…(omitted)
DESCRIPTION
 The cron utility is launched by launchd(8)…(omitted)

[CRONTAB]

NAME
 crontab — maintain crontab files for individual users (V3)
…(omitted)
DESCRIPTION
…(omitted)
(Darwin note: Although cron(8) and crontab(5) are officially supported under Darwin, their functionality has been absorbed into launchd(8), which provides a more flexible way of automatically executing commands. See launchctl(1) for more information.)

[LAUNCHD]

NAME
 launchd — System wide and per-user daemon/agent manager
DESCRIPTION
 launchd manages processes, both for the system as a whole and for individual users.
…(omitted)
During boot launchd is invoked by the kernel to run as the first process on the system and to further bootstrap the rest of the system.

[LAUNCHCTL]

NAME
 launchctl — Interfaces with launchd
…(omitted)
DESCRIPTION
 launchctl interfaces with launchd to manage and inspect daemons, agents and XPC services.

I know we all read through the above definitions word by word, with patience and grace. So here is a brief recap:

  1. Launchd is the manager of daemons/agents (both system-wide and per-user).
  2. Cron is a daemon to execute scheduled commands.
  3. When a job is assigned to cron, under the hood it is launchd that does the work.

Thus we deduct: Launchd manages cron, has the functionality of cron, and does the real job for cron.

Who says life is fair?

While we are questioning cron’s reason of existence[1], let us use launchd to prop up our script.

Launchctl

To communicate with launchd, we use the interface launchctl. Three steps are required: (1) Write a plist file (where all the configurations for this job are) (2) Place the plist file under ~/Library/LaunchAgents (3) Load the file.

(1) Write a plist file

https://gist.github.com/jenny-codes/3ad160580627c34f98f1dca173fb821f

Fill in the blanks, and we’re good to go.

In this plist, the program at /usr/local/bin/git_pushall, which we just wrote, will be run twice a day, respectively at 10:03 and 20:03. If this is your first time dealing with plist, you may need a little time to get used to (it took me some 20 minutes to figure this out–I’m sure you can do better). Basically the <key> is the field of the configuration option, and the next line is its value. Here is a list of the keys you can configure.

Also, run this line to check if your plist syntax is valid:

plutil -lint your_plist_file.plist

(2) Put the plist file into ~/Library/LaunchAgents

There are several locations we can place our plist files:

# Per-user agents provided by the user.
~/Library/LaunchAgents
# Per-user agents provided by the administrator.
/Library/LaunchAgents
# System-wide daemons provided by the administrator.
/Library/LaunchDaemons
# Per-user agents provided by Apple.
/System/Library/LaunchAgents
# System-wide daemons provided by Apple.
/System/Library/LaunchDaemons

In our case, we will put our file under ~/Library/LaunchAgents.

(3) Load the file

The commands we need:

# See the services list.
launchctl list
# Load your plist
launchctl load [file name]
# Unload the service
launchctl unload [file name]
# Start the service (immediately, regardless of schedule)
launchctl start [service label]
# remove ghost services (no longer used)
launchctl remove [service label]

We use launchctl load [file name] to load the plist file onto launchctl. Use launchctl list to see if it is loaded, and launchctl start [service label] for testing. If you make any changes after the upload, unload and load it again.

Note that some commands require [file name] while others require [service label], where you define in your plist under key label.

Voila! That’s pretty much it!

Note

[1] Apple’s document: ‘Although it is still supported, cron is not a recommended solution. It has been deprecated in favor of launchd.’

文章同步發表於 Medium


  • Find me at