Command line interface¶
Faculty has a command line client, faculty
. This lets you:
create, list, and terminate servers;
access your servers via SSH;
transfer large files using appropriate protocols like
scp
andrsync
.
faculty
is designed to be used from a terminal on your local machine. It will
work on macOS, Linux and Windows.
Note
The command line client was formerly called sml
. The API will remain
the same. The sml
client will no longer receive updates.
Installation on MacOS and Linux¶
You can install it using pip
:
$ pip install -U faculty-cli
If you don’t have pip
, follow these instructions. If you get an error
along the lines of Permission denied on macOS or Linux, you may need to
re-run this command with sudo
.
Once you have done this, continue to the Initialising faculty section.
Installation on Windows¶
The recommended way to use faculty
on Windows is via Cygwin. This has only been tested on Windows 10.
Make sure you have an installation of Cygwin with the following packages. Installing new packages normally involves restarting the Cygwin installer.
python3 (in the Interpreters, Python category)
openssh (in the Net category)
rsync (in the Net category)
In the Cygwin terminal, run:
$ python3 -m ensurepip
$ pip3 install -U faculty-cli
Once you have done this, continue to the Initialising faculty section.
Initialising faculty¶
The first time you use faculty
, you will need to configure it with a client
ID and secret. This is used to authenticate access to your resources in
Faculty, but is different from your usual username and password. You can create
client credentials through the My account page.
Once you’ve created your credentials, run the following and enter the credentials when prompted:
$ faculty login
Once faculty
is configured, you can use it to create a new server on a project,
sync data up to a project, and connect to a server from a terminal on your
local machine.
To see the full list of commands run:
$ faculty --help
While most of the Faculty functionality is accessible through the UI, there are some things that you can only do through the command line (for instance, bulk uploading code to your workspace).
List projects and servers¶
You can list your Faculty projects with:
$ faculty project list
and you can list a project’s running servers with:
$ faculty server list <project>
Note
faculty server list
, like many other faculty
subcommands, takes a
project as an argument. For projects with a unique name, you can
simply give the project name, however for projects with non-unique
names, you must pass the project ID. Project IDs are discoverable
using faculty project list -v
.
SSH into a server¶
Having successfully listed your servers, you can get SSH access using:
$ faculty shell <project> <server>
You can also provide faculty shell
with arbitrary extra arguments, which will
be passed through to ssh
. For example, to set up an SSH tunnel to an
arbitrary network port on the server:
$ faculty shell <project> <server> -L 9000:localhost:8888
When you SSH into a server, you will be in the /home/faculty
directory.
You probably want to access your project files first. These are located in
/project
. You can jump to that directory with:
$ cd /project
Note
As with projects, faculty shell
and other commands that take a
server as an argument can be passed either the server name or server
ID. If you have servers with non-unique names, use faculty server list
<project> -v
to discover their IDs for use in faculty shell
.
Create a new server¶
You can create new servers for a project using:
$ faculty server new <options> <project>
For example:
$ faculty server new --cores=8 --memory=16 --name=new-server user-playground
Full options can be listed with faculty server new --help
.
Note
If the name of the project is not unique among projects you have
access to, you must pass the project ID rather than the project
name. Project IDs are discoverable using faculty project list -v
.
Open the web interface of a server in your browser¶
You can open the web interface of a server in your browser with:
$ faculty server open <project>
faculty
will open the web interface of one of the running servers in the
project. If you wish to open the interface of a specific server, pass the
server’s name or ID with --server <server>
.
Copying files to and from your workspace¶
You can also use faculty
to upload and download files to and from your project
on the command line. To upload a file, use faculty file put
:
$ faculty file put <project> <local-source> <remote-destination>
Similarly, to download a file, use faculty file get
:
$ faculty file get <project> <remote-source> <local-destination>
These commands will copy files from the source path provided to the destination path.
You need to have a server running on the project to upload files. If you have
multiple servers, you can specify which server the files are routed through
with the --server
option.
Note
The remote path in faculty file put
and faculty file get
will be relative
to the Unix home directory (/home/faculty
) on the Faculty server.
To put files directly in the project workspace, you will want to use a
remote path under /project
.
faculty file put
and faculty file get
use the scp
command internally, and
any additional command line arguments provided on the command line will be
passed through to scp
. Advanced users may find this useful to customise the
behaviour of faculty file put
and faculty file get
.
Uploading large files to Faculty¶
When uploading large files, we recommend using faculty file sync-up
, as this
supports resuming the upload process if your connection drops. To start a new
upload with resume support, use the following command:
$ faculty file sync-up <project> path/on/local/machine path/on/faculty --partial --progress
To resume an unfinished upload:
$ faculty file sync-up <project> path/on/local/machine path/on/faculty --append --progress
Uploading and downloading a large number of files to and from Faculty¶
If you have an entire directory of files to upload to Faculty, you can do so
with faculty file sync-up
. To upload a directory:
$ faculty file sync-up <project> path/on/localmachine/ path/on/faculty
To download an entire directory from Faculty to your local machine use faculty
file sync-down
:
$ faculty file sync-down <project> path/on/faculty/ path/on/localmachine
You can also provide faculty file sync-down
and faculty file sync-up
with
arbitrary extra arguments, which will be passed through to the rsync
program. For example, to upload a directory with compression and delete files
in the directory on Faculty not present on your local machine, use:
$ faculty file sync-up <project> path/on/localmachine/ path/on/faculty --compress --delete
Note
Users unfamiliar with the rsync
program, and especially the
meaning of trailing slashes on the local and remote arguments, and
the meaning of the --delete
option, are strongly recommended to
read its documentation before using faculty file sync-down
or faculty
file sync-up
. Incorrect use of these features may result in data
loss.
Server environments¶
Server Environments can be created
in Faculty and applied through the web interface or through faculty
. The
faculty environment
subcommands provide the ability to list and apply
environments in a project, and to view the results of applying environments.
To list the environments in a project, run:
$ faculty environment list <project>
This will list all environments in the specified project.
You can apply one of these environments to an existing server using the faculty
environment apply
command:
$ faculty environment apply <project> <server> <environment>
As with other commands, each of the project, server and environment can be a
name or an ID. To get the ID of server environments in a project, you can use
the -v
option with faculty environment list
.
The status of the last environment applied to a server can be checked with the
faculty environment status
command:
$ faculty environment status <project> <server>
This will summarise the steps of the environment being applied, along with the
overall status of the execution. To stream the logs of the environment
application as it’s being applied, use the faculty environment logs
command:
$ faculty environment logs <project> <server>
You can also specify a particular step to stream the logs of using the
--step
command line option. This accepts the index of the step as given by
faculty environment status
.
Jobs¶
Jobs can be created in Faculty and run through the web interface or
through faculty
. The faculty job
subcommands provide the ability to list and
run jobs in a project, and to view the logs for a run.
To list the jobs in a project, run:
$ faculty job list <project>
This will list all jobs in the specified project.
You can run a job using the faculty job run
command:
$ faculty job run <project> <job>
The project and job can be a name or ID. To get the ID of job in a project, you
can use the -v
option with faculty job list
.
To run a job with parameters, run:
$ faculty job run <project> <job> "foo=bar,eggs=spam"
This will start a single job with the parameters “foo” and “eggs” set to “bar” and “spam” respectively.
To run a job multiple times with different parameters, run:
$ faculty job run <project> <job> "foo=bar1,eggs=spam1" "foo=bar2,eggs=spam2"
This will start two jobs, the first with parameters “bar1” and “spam1” and the second with parameters “bar2” and “spam2”.
You can also run a job multiple times with no parameters using the
--num-subruns
command line option.
To list the runs of a job, use the faculty job list-runs
command:
$ faculty job list-runs <project> <job>
You can view the logs for a run using:
$ faculty job logs <project> <job> <run number>
For example, to view the logs for the second run of a job, use:
$ faculty job logs <project> <job> 2
To view the logs for the fifth sub-run of the third run of a job, use:
$ faculty job logs <project> <job> 3.5
Managing the credentials for multiple deployments¶
By default, when you have completed the Initialising faculty section, there will
be a single default
set of credentials saved on your machine.
If you are working on multiple Platform deployments you have to manually edit the credentials
file at initialization and add the credentials for the additional deployments (each will generate you
a different set of credentials) The credentials are found at ~/.config/faculty/credentials
on MacOS/Linux/Cygwin), and you need to add another section, following the existing pattern as:
[profile-name]
domain = ...
client_id = ...
client_secret = ...
Here the value of profile-name
can be anything, just have to be unique in the file.
Afterwards you can switch which Platform deployment your CLI is interacting with by
setting the FACULTY_PROFILE
environment variable to the relevant profile-name
value. For example, to set this value for the current shell on MacOS/Linux:
$ export FACULTY_PROFILE=profile-name
after which the faculty
will interact with that profile for its operation.
Using the Faculty CLI from a Faculty server with multiple deployments¶
It is possible to use the Faculty CLI from a Faculty server to interact with a second Faculty deployment. For example, a usecase could be to transfer data between Faculty deployments.
First you will need to setup the Faculty CLI to use multiple deployments from within a Faculty server.
Ensure that
~/.config/faculty/credentials
contains a default profile for the current source deployment, and a named profile for the destination deployment.Clear the following environment variables from the Faculty server environment:
unset FACULTY_CLIENT_ID unset FACULTY_CLIENT_SECRET unset FACULTY_PROTOCOL unset FACULTY_DOMAIN unset SHERLOCKML_CLIENT_ID unset SHERLOCKML_CLIENT_SECRET unset SHERLOCKML_PROTOCOL unset SHERLOCKML_DOMAIN
Override the Faculty CLI credentials to use the destination deployment profile on a single line.
The following command will list projects from the destination deployment:
FACULTY_PROFILE=<Destination Profile> faculty project list
To transfer a local workspace file from the source Faculty server, to a project dataset on a destination Faculty deployment:
FACULTY_PROFILE=<Destination Profile> faculty datasets put <Destination Project> <Local File> <Destination Dataset Path>