Wcscmd

wcscmd is a set of command line management tools developed based on python, supporting python2 , python3 and common operating systems

Install

It is recommended to use pip to install

• direct installation

pip install wcs-python-sdk

• update method

pip install -U wcs-python-sdk

initialization

Before using the SDK, you need to obtain a pair of valid AccessKey and SecretKey signature authorization.

It can be obtained by the following methods:

1. Open a Wangsu Cloud Storage (WCS) account
2. Log in to the Wangsu SI platform, and view the AccessKey and SecretKey in Security Management - Secret Key Management
3. Log in to the Wangsu SI platform, and check the uploaded Domain name(puturl) and the managed Domain name (mgrurl) in Security Management - Domain name Management.

After obtaining the above four configurations, execute the following command to initialization the configuration information through command line interaction:
wcscmd --configure

The updated configuration information will be saved in the .wcscfg file in the $HOME directory, and the configuration information added in the previous step can be printed by the following command:
wcscmd --dump-config

The configuration Parameter in the .wcscfg file are described as follows:

access\_key #user ak  
block\_size chunk uploading block size, the default value is 4194304, no unit is required when configuring, the default unit is B  
bput\_retries chunk uploading, bput request retransmission times  
bput\_size #The size of the piece in the chunk uploading block, the default value is 524288, no unit is required when configuring, the default unit is B  
callbackBody #After the callback is uploaded successfully, the data submitted by the server to callbackurl  
callbackUrl #After the callback is uploaded successfully, the server requests the address by POST  
concurrency #The block Concurrency of chunk uploading , when the Concurrency is set to 0, it is uploaded sequentially  
connection\_retries #The number of retransmissions when requesting to establish a connection  
connection\_timeout #The timeout time for requesting to establish a connection, in seconds  
contentDetect #After the file is uploaded successfully, perform content identification operation  
detectNotifyRule #Authentication result notification rule setting  
detectNotifyURL #Receive the notification address of the authentication result, which must be a public URL address  
force #Enforce data processing, the default value is 0, do not enforce data processing and overwrite the original result  
ishttps #Whether to use https to initiate the request  
limit #List the limit Parameter of the resource API, configure the list items  
marker #List the maker Parameter of the resource API, configure the position marker returned by the last listing, and use it as the starting point information for this listing  
mgr\_url #User's management Domain name  
mkblk\_retries #Piece chunk uploading, mkblk request retransmission times  
mkfile\_retries #Piece chunk uploading, mkfile request retransmission times  
mode #List the mode Parameter of the resource API, and configure the sorting method of the list  
notifyurl #Asynchronous processing API processing result notification receiving URL  
output #Save the description information of the task processing result to the specified file, the format is: <bucket>:<key>.  
overwrite #Whether to overwrite when the upload API finds that the file already exists  
persistentNotifyUrl #The address to receive the preprocessing result notification  
persistentOps #After the file is uploaded successfully, the list of preprocessing instructions  
prefix #List the prefix Parameter of the resource API  
put\_url #User upload Domain name  
returnBody #After the upload is successful, customize the data that is finally returned to the uploader  
returnUrl #After the upload is successful, the server requests the address by POST  
secret\_key #user sk  
separate #Whether the processing instructions are notified separately  
tmp\_record\_folder chunk uploading upload progress record directory  
upload\_id #The task id of chunk uploading Connection resuming on break-point

The wcscmd command line tool uses

Windows system needs to add python to execute commands, such as python wcscmd --help

View tool instructions

wcscmd --help  
Commands:  
List objects support Parameter such as –prefix aa  
wcscmd list wcs://BUCKET RESFILE  
List buckets  
wcscmd listbucket  
download file  
wcscmd get URL  
Delete a file  
wcscmd del wcs://BUCKET/OBJECT  
Move a file from src bucket to des bucket  
wcscmd mv wcs://srcBUCKET/srcOBJECT wcs://dstBUCKET/desOBJECT  
Copy a file from src bucket to des bucket  
wcscmd cp wcs://srcBUCKET/srcOBJECT wcs://dstBUCKET/desOBJECT  
Set deadline of file  
wcscmd setdeadline wcs://BUCKET/OBJECT deadline  
Get file info  
wcscmd stat wcs://BUCKET/OBJECT  
Upload a local file to WCS  
wcscmd put wcs://BUCKET/OBJECT LOCALFILE  
Multipart upload a local file to WCS  
wcscmd multiput wcs://BUCKET/OBJECT LOCALFILE  
Delete multiple files according to prefix  
wcscmd deletePrefix wcs://BUCKET PREFIX  
Fops audio/video processing  
wcscmd fops wcs://BUCKET/OBJECT fopsparm  
Get fops task results  
wcscmd fopsStatus persistentId  
Get fmgr task results  
wcscmd fmgrStatus persistentId

wcscmd-common upload

The upload Policy can be defined by editing the corresponding configuration items in the .wcscfg file, or it can be temporarily configured through the option of the command line,

wcscmd put wcs://BUCKET/OBJECT localPath --overwrite 1

wcscmd- chunk uploading

The upload Policy can be defined by editing the corresponding configuration items in the .wcscfg file, or it can be temporarily configured through the option of the command line. If you need to Connection resuming on break-point, you need to add the option –upload-id, the priority of the upload-id higher than the upload id configured in .wcscfg

wcscmd multiput wcs://BUCKET/OBJECT localPath --upload-id 3IL3ce3kR6kDf4sihxh0LcWUpzTYEKFf

wcscmd - enumerate space list

wcscmd listbucket

wcscmd - enumerate space file list

The enumeration results of the space test will be saved in the result file in the current directory

wcscmd list wcs://BUCKET ./result --limit 4 --marker 2

wcscmd download file

Without the filename Parameter, the downloaded file will have the same name as the Source file by default, and it will be saved in the current directory. With the filename Parameter, the downloaded file will be saved in the current directory, and the file name will be the Parameter filename
url needs to be enclosed in ‘’ quotes

wcscmd get [URL] [filename]

wcscmd - get file information

wcscmd stat wcs://BUCKET/OBJECT

wcscmd - set file retention period

The storage time unit is days, 0 means to delete as soon as possible, -1 means to cancel the expiration time, and save forever. When you want to set -1, you need to include the whole in quotation marks

wcscmd setdeadline wcs://BUCKET/OBJECT 3 wcscmd setdeadline wcs://BUCKET/OBJECT '"-1"'

wcscmd - delete file

wcscmd del wcs://BUCKET/OBJECT

wcscmd - delete files by prefix

wcscmd deletePrefix wcs://BUCKET test-prefix

wcscmd - move files

wcscmd mv wcs://SRCBUCKET/SRCOBJECT wcs://DSTBUCKET/DSTOBJECT

wcscmd - copy files

wcscmd cp wcs://SRCBUCKET/SRCOBJECT wcs://DSTBUCKET/DSTOBJECT

Calculate file etag value

wcs-python-sdk provides a tool to calculate the etag value of a file, and users can experience this function through the command line

/usr/bin/wcs_etag_cal -h usage: WCS-Python-SDK [-h] {etag} ... positional arguments: {etag} etag etag [file...] optional arguments: -h, --help show this help message and exit /usr/bin/wcs_etag_cal etag filepath1 filepath2 [filepath1, filepath2] FrA377uGHSxcTM62-rjsjvoKqRVS FiUsqBkZ6e8KaAA9Uu6q3qLPgmDW

S3cmd

s3cmd is an S3 command-line tool based on Linux and Mac OS operating systems.

configuration steps

1. Download the installation package, download link: https://sourceforge.net/projects/s3tools/files/s3cmd/
2. Unzip the installation package, tar -xzvf s3cmd-1.6.1.tar.gz
3. Move path, mv s3cmd-1.6.1 /usr/local/s3cmd - create soft connection, ln -s /usr/local/s3cmd/s3cmd /usr/bin/s3cmd
4. Execute the configuration command (enter the corresponding password according to the prompt), s3cmd --configure, after executing this command, the ~/.s3cfg configuration file will be generated.
5. Edit the configuration file and modify the following Parameter: vi ~/.s3cfg

access_key = 7e844d6xxxxe8e320b627f7fc8baf9c501a secret_key = ec471c7fxxxx769c73dbb83a6767e7d2791 host_base = 127.0.0.1:7085

Instructions

Use the s3cmd command line tool, enter s3cmd –h, and the help instructions will be displayed.

list buckets

Order:

s3cmd ls

Note: The s3cmd tool uses the v4 signature version by default. To use the v2 version, add the --signature-v2 option.

delete bucket

Order:

s3cmd rb s3://mybucket

list objects

s3cmd ls s3://bucket\[/PREFIX\]

Note: To list the files and folders in the bucket, the file will display the modification time and size, and the directory will display DIR.

delete object

s3cmd del s3://BUCKET/OBJECT或s3cmd rm s3://BUCKET/OBJECT

normal upload

s3cmd put FILE [FILE...] s3://BUCKET[/PREFIX]

When the size of the uploaded file is smaller than the size set in the configuration file, the normal upload method will be used.

chunk uploading

s3cmd put FILE [FILE...] s3://BUCKET[/PREFIX]

When the size of the uploaded file is larger than the size set in the configuration file, the chunk uploading method will be used, and the size of each block is configured according to the configuration file.

download object

Download a single file:

s3cmd get s3://BUCKET/OBJECT LOCAL_FILE

Recursively download all files in a directory:

s3cmd get --recursive s3://BUCKET/dir/* ./

copy object

s3cmd cp s3://BUCKET1/OBJECT1 s3://BUCKET2[/OBJECT2]

delete multiple objects

s3cmd supports the operation of deleting multiple objects, which is to send multiple deletion request commands at the same time. The delete command is as follows:

s3cmd del s3://bucket/object s3://bucket/object

synchronous operation

s3cmd sync LOCAL\_DIR s3://BUCKET\[/PREFIX\] or s3://BUCKET\[/PREFIX\] LOCAL\_DIR  
Synchronize local files to the target bucket or sync files from the bucket to the local. MD5 verification is required for the synchronization operation, and only when the files are different will they be transmitted.

Synchronize all files in the current directory:  
s3cmd sync ./ s3://bucket/

After adding the "–dry-run"Parameter, only the items that need to be synchronized are listed, and no actual synchronization is performed:  
s3cmd sync --dry-run ./ s3://bucket/

After adding the "--delete-removed"Parameter, files that do not exist locally will be deleted:  
s3cmd sync --delete-removed ./ s3://bucket/

After adding the "--skip-existing"Parameter, the MD5 verification will not be performed, and the existing local files will be skipped directly:  
s3cmd sync --skip-existing ./s3://my-bucket-name/