Server functionality

The main purpose of SFTP.jl is to connect to a server with the secure file transfer protocol and exchange files between the server and the local system.

Connecting to the server

SFTP.jl takes care of the connection to the server all by itself. All that needs to be done is to instantiate a SFTP.Client with one of the appropriate constructors for either authentication with username and password or with a certificate.

SFTP.Client — Type

mutable struct SFTP.Client

SFTP manages the connection to the server and stores all relevant connection data.

Fields

downloader::Downloader: for handling downloads and managing connections, name lookups, and other resources
uri::URI: save the URI including the present path on the sftp server
username::String: mandatory user name
password::String: optional password, for access by username/password
disable_verify_peer::Bool: disable verification of the peer's SSL certificate
disable_verify_host::Bool: disable verification of the certificate's name against host
verbose::Bool: set Curl verbosity
public_key_file::String: the public key file of the certificate authentication
private_key_file::String: the private key file of the certificate authentication

Constructors

SFTP.Client(url::AbstractString, username::AbstractString, public_key_file::AbstractString, public_key_file::AbstractString; kwargs) -> SFTP.Client
SFTP.Client(url::AbstractString, username::AbstractString, password::AbstractString=""; kwargs) -> SFTP.Client

Note

A username must be provided for both methods to work.

Setup certificate authentication

Before using the constructor method for certificate authentication, private and public key files must be created and stored in the ~/.ssh folder and on the server and the local system, respectively, e.g., ~/.ssh/idrsa and ~/.ssh/idrsa.pub. Additionally, the host must be added to the known_hosts file in the ~/.ssh folder.

Testing certificate authentication

The correct setup can be tested in the terminal with ssh myuser@mysitewhereIhaveACertificate.com.

Construct an SFTP.Client from the url and either user information or public and private key files.

Arguments

url: The url to connect to, e.g., sftp://mysite.com
username/password: user credentials
public_key_file/public_key_file: authentication certificates

Keyword arguments

The following keyword arguments exist with default values given in parentheses:

create_known_hosts_entry: Automatically create an entry in known hosts
disable_verify_peer: verify the authenticity of the peer's certificate
disable_verify_host: verify the host in the server's TLS certificate
verbose: display a lot of verbose curl information

Examples

sftp = SFTP.Client("sftp://mysitewhereIhaveACertificate.com", "myuser", "test.pub", "test.pem")
sftp = SFTP.Client("sftp://mysitewhereIhaveACertificate.com", "myuser")
sftp = SFTP.Client("sftp://test.rebex.net", "demo", "password")

source

File exchange with the server

Use the upload and download functions to exchange data with the SFTP server. Options exist for conflicts with already existing files to throw an error, skip the exchange or force an overwrite. The functions can be used to exchange single files, exchange directories recursively or broadcast over a number of files or folders.

For download, an additional method exists to save contents from a remote file directly to a variable.

SFTP.upload — Function

upload(sftp::Client,  src::AbstractString=".", dst::AbstractString="."; kwargs...) -> String

Upload (put) src to dst on the sftp server; src can be a file or folder. Folders are uploaded recursively. dst must be an existing folder on the server, otherwise an IOError is thrown. src may include an absolute or relative path on the local system, which is ignored on the server. dst can be an absolute path or a path relative to the current uri path of the sftp server. The function returns dst as String.

see also: download/ download method

Keyword arguments

merge::Bool=false: download into existing folders, when true or throw an IOError
force::Union{Nothing,Bool}=nothing: Handle conflicts of existing files/folders
- nothing (default): throw IOError for conflicts
- true: overwrite existing files/folders
- false: skip existing files/folders
- if merge = true, force counts only for files
ignore_hidden::Bool=false: ignore hidden files and folders, if true
hide_identifier::Union{Char,AbstractString}='.': start sequence of hidden files and folders

Examples

sftp = SFTP.Client("sftp://test.rebex.net", "demo", "password")

upload(sftp, "data/test.csv", "/tmp") # upload data/test.csv to /tmp/test.csv

files=readdir()
upload.(sftp, files) # upload the current directory to the current directory on the server

upload(sftp, ignore_hidden=true) # the current folder is uploaded to the server without hidden objects

source

Base.download — Function

download(sftp::Client, src::AbstractString=".", dst::AbstractString="."; kwargs...) -> String

Download src from the sftp server to dst on the local system; src can be a file or folder. Folders are downloaded recursively. dst must be an existing folder on the local system, otherwise an IOError is thrown. src may include an absolute or relative path on the sftp server, which is ignored on the local system. dst can be an absolute or relative path on the local system. The function returns dst as String.

see also: upload, other download method

Keyword arguments

merge::Bool=false: download into existing folders, when true or throw an IOError
force::Union{Nothing,Bool}=nothing: Handle conflicts of existing files/folders
- nothing (default): throw IOError for conflicts
- true: overwrite existing files/folders
- false: skip existing files/folders
- if merge = true, force counts only for files
ignore_hidden::Bool=false: ignore hidden files and folders, if true
hide_identifier::Union{Char,AbstractString}='.': start sequence of hidden files and folders

Example

sftp = SFTP.Client("sftp://test.rebex.net/pub/example/", "demo", "password")
files=readdir(sftp)
download_dir="/tmp"
download.(sftp, files, download_dir)

Alternatively:

sftp = SFTP.Client("sftp://test.rebex.net/pub/example/", "demo", "password")
donwload(sftp) # downloads current folder on server to current directory on local system

source

Base.download — Method

download(fcn::Function, sftp::SFTP.Client, src::AbstractString)

Download src from the sftp server and use fcn to retrieve the data from src. src may include an absolute or relative path to a file on the sftp server. Only temporary files are created on the local system and directly deleted after data reading.

see also: upload, other download method

Defining read functions

fcn must have one AbstractString parameter, which is used to pass the temporary file path to any function that reads the contents. The function must return the contents of the file in any desired format, e.g. Matrix, DataFrame or array.

Examples

using CSV
# Define functions to process the file data
fcsv(path::AbstractString)::Matrix{Int} = CSV.read(path, CSV.Tables.matrix)
fread(path::AbstractString)::Vector{String} = readlines(path)
# Download data to variable
matrix = download(fcsv, sftp, "data/matrix.csv")
array = download(fread, sftp, "data/matrix.csv")

source