You have reached a page that used to be the MinIO console.
⚠ Use path-style addressing
(https://s3.smtornado.com/bucket/key), not virtual-host style
(https://bucket.s3.smtornado.com/key). Some clients default to
virtual-host and fail with a DNS error that looks like the endpoint is
down.
Use your newly issued Garage access key and secret. Old MinIO keys do not work. Each credential has its own reviewed bucket permissions.
mc)mc alias set lab https://s3.smtornado.com ACCESS_KEY SECRET_KEY --api S3v4
mc ls lab/
mc ls lab/datasets
mc cp lab/datasets/table.csv .
mc mirror lab/datasets ./datasets # a whole prefix
The same mc you used with MinIO. Only the alias
changes. ⚠ mc admin commands do not work — those are MinIO
server administration and Garage is a different server.
aws configure --profile lab
# Access Key ID : ACCESS_KEY
# Secret Access Key : SECRET_KEY
# Default region : lab
# Default output : json
aws --profile lab --endpoint-url https://s3.smtornado.com s3 ls
aws --profile lab --endpoint-url https://s3.smtornado.com s3 ls s3://datasets/
aws --profile lab --endpoint-url https://s3.smtornado.com \
s3 cp s3://datasets/table.csv .
⚠ --endpoint-url on every command, or the CLI
talks to Amazon. To avoid repeating it, put
endpoint_url = https://s3.smtornado.com under
[profile lab] in ~/.aws/config — supported by
recent AWS CLI v2.
import boto3
from botocore.config import Config
s3 = boto3.client(
"s3",
endpoint_url="https://s3.smtornado.com",
aws_access_key_id="ACCESS_KEY",
aws_secret_access_key="SECRET_KEY",
region_name="lab",
# path-style addressing — see the note above
config=Config(s3={"addressing_style": "path"},
signature_version="s3v4"),
)
for page in s3.get_paginator("list_objects_v2").paginate(
Bucket="datasets", Prefix=""):
for obj in page.get("Contents", []):
print(obj["Key"], obj["Size"])
s3.download_file("datasets", "table.csv", "table.csv")
Keep the credentials out of the source file — boto3 reads
~/.aws/credentials, or the environment
(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY), if you
drop the two arguments.
mc,
aws, boto3, rclone
works⚠ If you have code doing unsigned S3 calls —
boto3 with UNSIGNED, or a bucket you made public
with mc anonymous — it will fail against the new store, and no
change of URL fixes it. Garage implements neither bucket policies nor ACLs;
permission is per access key. Either use a key, or use the public HTTPS links
below, which serve the same objects over ordinary anonymous HTTP.
Datasets that were published with a public link still resolve
at the URLs they were published at, anonymously, in a browser and with
curl and wget. Those links were not changed and will
not be.
mc cp ./big.fits lab/datasets/
aws --profile lab --endpoint-url https://s3.smtornado.com \
s3 cp ./big.fits s3://datasets/
⚠ Large uploads go straight to the origin rather than through
the CDN, so a single request is not capped at 100 MB. If you see
413 on an upload, your client is not using this endpoint — check
for a proxy or an old alias.