GitHub

Filesystem functions

SFTP.jl overloads a number of Julia's Base Filesystem functions to navigate and manipulate server paths and retrieve stats on path objects (files, symlinks or folders). The SFTP filesystem methods mimic Julia's Filesystem functions or Linux filesystem functions, respectively, however, SFTP.jl's functionality might be reduced.

URI

Several of the Filesystem functions return a URI struct from the URIs package to represent the complete server address including the path on the server. For easier handling, the URI struct is exported by SFTP.jl as well. See URIs' documentation for more details on the URI struct.

Base.Filesystem.cd — Method
cd(sftp::SFTP.Client, dir::AbstractString)

Set the current working directory as dir in the uri of the sftp client.

see also: pwd, mv, rm

source
Base.Filesystem.mv — Method
mv(
    sftp::SFTP.Client,
    src::AbstractString,
    dst::AbstractString;
    force::Bool=false
)

Move src to dst on the sftp server. dst must be moved to an existing parent folder; src is overwritten without warning, if force is set to true.

see also: pwd, cd, rm

source
Base.Filesystem.rm — Method
rm(sftp::SFTP.Client, path::AbstractString; recursive::Bool=false, force::Bool=false)

Remove (delete) the path on the sftp server. Set the recursive flag to remove folders recursively. Suppress errors by setting force to true.

Warning

Recursive deletions can be very slow for large folders.

see also: pwd, cd, mv

source
Base.Filesystem.mkdir — Method
mkdir(sftp::SFTP.Client, dir::AbstractString) -> String

Create a new dir on the sftp server and return the name of the created directory. Although a path can be given as dir, dir can only be created in an existing directory, i.e. the path up to the basename of dir must exist. Otherwise, and in case of already existing folders, an error is thrown.

see also: mkpath

source
Base.Filesystem.mkpath — Method
mkpath(sftp::SFTP.Client, path::AbstractString) -> String

Create a path including all missing intermediate folders on the sftp server and return path as String on success. No errors are thrown for existing paths.

See also: mkdir

source
Base.Filesystem.readdir — Method
readdir(
    sftp::Client,
    path::AbstractString=".";
    join::Bool=false,
    sort::Bool=true,
    check_path::Bool=false
) -> Vector{String}

Return the names of all objects in the directory dir (or the current working directory if not given) on the sftp server. If join is set to true, the list of file names includes the absolute paths. Sorting of file names can be switched off with the sort flag to optimise performance. Depending on the server settings, readdir may return an empty vector for non-existant paths. To ensure an error is thrown for non-existant paths, set check_path to true.

Note

Setting check_path to true can drastically reduce the performance for large folders. If you know the folder structure, you should avoid setting this flag.

see also: walkdir

source
Base.Filesystem.walkdir — Method
walkdir(
    sftp::Client,
    root::AbstractString=".";
    topdown::Bool=true,
    follow_symlinks::Bool=false,
    skip_restricted_access::Bool=true,
    sort::Bool=true,
    ignore_hidden::Bool=false,
    hide_identifier::Union{AbstractString,Char}='.'
) -> Channel{Tuple{String,Vector{String},Vector{String}}}

Return an iterator that walks the directory tree of the given root on the sftp server. If the root is omitted, the current URI path of the sftp server is used. The iterator returns a tuple containing (rootpath, dirs, files). The directory tree can be traversed top-down (topdown=true) or bottom-up (topdown=false).

If follow_symlinks is set to true, the sources of symlinks are listed rather than the symlink itself as a file. If sort is set to true, the files and directories are listed alphabetically. Hidden paths starting with the hide_identifier sequence can be skipped by setting ignore_hidden to true.

If a remote folder has restricted access, these directories are skipped with an info output on the terminal unless skip_restricted_access is set to false, in which case an Downloads.RequestError is thrown.

see also: readdir

Examples

sftp = SFTP.Client("sftp://test.rebex.net/pub/example/", "demo", "password")
for (root, dirs, files) in walkdir(sftp, "/")
    println("Directories in $root")
    for dir in dirs
        println(joinpath(root, dir)) # path to directories
    end
    println("Files in $root")
    for file in files
        println(joinpath(root, file)) # path to files
    end
end
source

Analyse and manipulate server paths

Known issue

URIs' joinpath method currently assigns two leading slashes (//) instead of one, if an absolute path is assigned to an empty URI path. This issue is addressed in PR #62.

Base.Filesystem.joinpath — Function
joinpath(sftp::SFTP.Client, path::AbstractString...) -> URI

Join any path with the uri of the sftp server and return an URI with the updated path. Any path components prior to an absolute path are dropped.

Note

The uri field of the sftp client remains unaffected by joinpath. Use sftp.uri = joinpath(sftp, "new/path", "parts") or cd(sftp, joinpath(sftp, joinpath("new/path", "parts"))) to update the URI on the sftp server.

source
Base.Filesystem.splitdir — Function
splitdir(sftp::SFTP.Client, path::AbstractString=".") -> Tuple{URI,String}

Join the path with the path of the URI in sftp and then split it into the directory name and base name. Return a Tuple of URI with the split path and a String with the base name.

Note

The returned path in the URI always contains a trailing slash while the basename is always without a slash.

Difference to Julia's splitdir method

splitdir splits the last non-empty path part from the remaining path. In contrast to Julia's splitdir method, trailing slashes do not result in an empty base name, but return the base name before the trailing slash.

see also: basename

source
Base.Filesystem.dirname — Function
dirname(sftp::SFTP.Client, path::AbstractString=".") -> String

Get the directory part of a path on the sftp server.

source
Base.Filesystem.basename — Function
basename(sftp::SFTP.Client, path::AbstractString=".") -> String

Get the file name or current folder name of a path. The path can be absolute or relative to the uri itself or of the sftp server given. If no path is given, the current path from the uri or sftp server is taken.

Difference to Julia's basename method

In contrast to Julia's basename, trailing slashes in paths are ignored and the last non-empty part is returned, except for the root, where the basename is empty.

see also: splitdir

source

Getting statistics on path objects

The SFTP.StatStruct holds all relevant information on server path objects. It can be directly analysed by functions analysing the filemode (filemode, isdir, isfile or islink). For these functions, additional convenience methods exist, which take the SFTP.Client and an AbstractString of the path as input arguments.

Tip

statscan should be preferred, whenever several objects are analysed in the same folder. Further analysis should be performed on the SFTP.StatStruct rather than with the convenience functions to improve performance. The convenience functions are mainly for single operations or when interactively exploring a server, but can mean significantly longer processing times for large folders on the server. Internally, the whole folder content is always scanned and the stats for the desired files are then retrieved from all scans.

SFTP.StatStruct — Type
struct SFTP.StatStruct

Hold information for file system objects on a Server.

Fields

  • desc::String: file or folder description/name
  • mode::UInt: file system object type (file, folder, etc.)
  • nlink::Int: number of hard links (contents)
  • uid::String: numeric user ID of the owner of the file/folder
  • uid::String: numeric group ID (gid) for the file/folder
  • size::Int64: file/folder size in Byte
  • mtime::Float64: modified time

Constructors

SFTP.StatStruct(stats::AbstractString) -> SFTP.StatStruct

Parse the stats string and return an SFTP.StatStruct.

The stats are of the format:

"d--x--x---  151 ftp      ftp          8192 Dec  2  2023 .."
source
SFTP.statscan — Function
statscan(
    sftp::SFTP.Client,
    path::AbstractString=".";
    sort::Bool=true,
    show_cwd_and_parent::Bool=false
) -> Vector{SFTP.StatStruct}

Like stat, but returns a Vector of SFTP.StatStruct with filesystem stats for all objects in the given path.

Tip

This should be preferred over stat for performance reasons.

Note

You can only run this on directories.

By default, the SFTP.StatStruct vector is sorted by the descriptions (desc fields). For large folder contents, sort can be set to false to increase performance, if the output order is irrelevant. If show_cwd_and_parent is set to true, the SFTP.StatStruct vector includes entries for "." and ".." on position 1 and 2, respectively.

see also: stat

source
Base.stat — Method
stat(sftp::SFTP.Client, path::AbstractString=".") -> SFTP.StatStruct

Return a SFTP.StatStruct with information about the path (file, directory or symlink) on the sftp server.

Note

This returns only stat data for one object, but stat data for all objects in the same folder is obtained internally. If you need stat data for more than object in the same folder, use statscan for better performance and reduced connections to the server.

see also: statscan

source
Base.Filesystem.isdir — Function
isdir(sftp::SFTP.Client, path::AbstractString=".") -> Bool
isdir(st::SFTP.StatStruct) -> Bool

Analyse the SFTP.StatStruct and return true for a directory, false otherwise. A convenience method exists to directly check the path on the sftp server. However, if several path objects in the same folder are analysed, it is much more performant to use statscan once and then analyse each SFTP.StatStruct.

see also: filemode, ispath, isfile, islink

source
Base.Filesystem.isfile — Function
isfile(sftp::SFTP.Client, path::AbstractString=".") -> Bool
isfile(st::SFTP.StatStruct) -> Bool

Analyse the SFTP.StatStruct and return true for a file, false otherwise. A convenience method exists to directly check the path on the sftp server. However, if several path objects in the same folder are analysed, it is much more performant to use statscan once and then analyse each SFTP.StatStruct.

see also: filemode, ispath, isdir, islink

source
Base.Filesystem.islink — Function
islink(sftp::SFTP.Client, path::AbstractString=".") -> Bool
islink(st::SFTP.StatStruct) -> Bool

Analyse the SFTP.StatStruct and return true for a symbolic link, false otherwise. A convenience method exists to directly check the path on the sftp server. However, if several path objects in the same folder are analysed, it is much more performant to use statscan once and then analyse each SFTP.StatStruct.

see also: filemode, ispath, isdir, isfile

source