What is certain now is that there is no problem using nginx to proxy the minio service (this configuration NGINX reverse proxy),
But I encountered some problems using traefik proxy minio, as follows:
When using mc cp minio/bucket minio2/bucket2, the following error message appears:
mc: <DEBUG> GET /agile-service/1/5724918fde504e3184ef3f2d161a3d8a@MYSQL.sql HTTP/1.1
Host: minio.xxx.x
User-Agent: MinIO (linux; amd64) minio-go/v7.0.70 mc/RELEASE.2024-05-24T09-08-49Z
Accept-Encoding: identity
Authorization: AWS4-HMAC-SHA256 Credential=**REDACTED**/20240604/us-east-1/s3/aws4_request, SignedHeaders=host;if-match;x-amz-content-sha256;x-amz-date, Signature=**REDACTED**
If-Match: "4ac555f86052cc0f34c993fce5d6b93b-1"
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20240604T092831Z
mc: <DEBUG> HTTP/1.1 200 OK
Content-Length: 1042
Accept-Ranges: bytes
Connection: keep-alive
Content-Security-Policy: block-all-mixed-content
Content-Type: application/x-sql
Date: Tue, 04 Jun 2024 09:28:43 GMT
Etag: "4ac555f86052cc0f34c993fce5d6b93b-1"
Last-Modified: Fri, 12 Jan 2024 12:37:37 GMT
Vary: Origin
X-Amz-Request-Id: 17D5C2EDEBDA9E80
X-Xss-Protection: 1; mode=block
mc: <DEBUG> Response Time: 76.7935ms
mc: <ERROR> Failed to copy `http://minio.xxx.x/agile-service/1/5724918fde504e3184ef3f2d161a3d8a@MYSQL.sql`. Put "http://minio2.xxx.x/agile-service/5724918fde504e3184ef3f2d161a3d8a@MYSQL.sql": net/http: HTTP/1.x transport connection broken: http: ContentLength=1042 with Body length 0
(3) cp-main.go:478 cmd.doCopySession(..) Tags: [http://minio.xxx.x/agile-service/1/5724918fde504e3184ef3f2d161a3d8a@MYSQL.sql]
(2) common-methods.go:508 cmd.uploadSourceToTargetURL(..) Tags: [http://minio.xxx.x/agile-service/1/5724918fde504e3184ef3f2d161a3d8a@MYSQL.sql]
(1) common-methods.go:212 cmd.putTargetStream(..) Tags: [minio2, http://minio2.xxx.x/agile-service/5724918fde504e3184ef3f2d161a3d8a@MYSQL.sql]
(0) client-s3.go:1225 cmd.(*S3Client).Put(..)
Commit:a8fdcbe7cb2f | Release-Tag:RELEASE.2024-05-24T09-08-49Z | Host:Ubuntu | OS:linux | Arch:amd64 | Lang:go1.22.3 | Mem:6.6 MiB/19 MiB | Heap:6.6 MiB/11 MiB
I discovered the cause of the problem, which is that there will be problems if some Chinese characters appear in my .sql file, whether it is .txt, .json, .sql, etc.
minio uses nginx proxy and runs in k8s cluster
minio2 uses traefik agent and runs in docker swarm cluster
I try to convert the file in minio
mc cp minio/agile-service/1/5724918fde504e3184ef3f2d161a3d8a@MYSQL.sql ./5724918fde504e3184ef3f2d161a3d8a@MYSQL.sql
Then use mc cp ./5724918fde504e3184ef3f2d161a3d8a@MYSQL.sql minio2/agile-service/1/5724918fde504e3184ef3f2d161a3d8a@MYSQL.sql
My traefik configuration and MinIO configuration are as follows:
version: "3.3"
services:
traefik:
image: traefik:v2.11.2
ports:
- "80:80"
command:
- --api.insecure=false # set to 'false' on production
- --api.dashboard=true # see https://docs.traefik.io/v2.11/operations/dashboard/#secure-mode for how to secure the dashboard
- --api.debug=true # enable additional endpoints for debugging and profiling
- --log.level=DEBUG # debug while we get it working, for more levels/info see https://docs.traefik.io/observability/logs/
- --log.format=json
- --accesslog=true
- --accesslog.format=json
- --providers.docker=true
- --providers.docker.swarmMode=true
- --providers.docker.exposedbydefault=false
- --providers.docker.network=public
- --entryPoints.web.address=:80
- --entryPoints.web.transport.respondingTimeouts.readTimeout=0 # https://doc.traefik.io/traefik/v2.11/routing/entrypoints/#respondingtimeouts
- --entryPoints.web.transport.respondingTimeouts.idleTimeout=0
volumes:
- /var/run/docker.sock:/var/run/docker.sock
networks:
- public
deploy:
resources:
limits:
cpus: '4'
memory: 8g
reservations:
cpus: '1'
memory: 2g
replicas: 1
restart_policy:
condition: on-failure
placement:
constraints:
- node.role == manager
minio:
container_name: minio
restart: always
image: minio:RELEASE.2021-06-07T21-40-51Z
command: minio server /data
expose:
- '9000'
environment:
- MINIO_ACCESS_KEY=xxx
- MINIO_SECRET_KEY=xxxx
- SET_CONTAINER_TIMEZONE=true
- CONTAINER_TIMEZONE=Asia/Shanghai
volumes:
- 'minio_data:/data'
- /etc/localtime:/etc/localtime:ro
networks:
- public
deploy:
labels:
- "traefik.enable=true"
- "traefik.http.routers.minio.rule=Host(`minio.xxx.x`)"
- "traefik.http.routers.minio.service=minio"
- "traefik.http.routers.minio.entrypoints=web"
- "traefik.http.services.minio.loadbalancer.server.port=9000"
- "traefik.http.services.minio.loadbalancer.passhostheader=true"
Add some error information
MINIO SERVER：
API: PutObject(bucket=agile-service, object=1/5724918fde504e3184ef3f2d161a3d8a@MYSQL.sql)
Time: 17:34:42 CST 06/04/2024
DeploymentID: 7ba2e7e3-3cef-4848-a2e7-a259032c9d9a
RequestID: 17D5C3677E890424
RemoteHost: 10.0.0.3
Host: minio2.xxx.x
UserAgent: MinIO (linux; amd64) minio-go/v7.0.70 mc/RELEASE.2024-05-24T09-08-49Z
Error: unexpected EOF (*errors.errorString)
5: cmd/fs-v1-helpers.go:316:cmd.fsCreateFile()
4: cmd/fs-v1.go:1128:cmd.(*FSObjects).putObject()
3: cmd/fs-v1.go:1044:cmd.(*FSObjects).PutObject()
2: cmd/object-handlers.go:1701:cmd.objectAPIHandlers.PutObjectHandler()
1: net/http/server.go:2069:http.HandlerFunc.ServeHTTP()
TRAEFIK：
2024-06-05T01:16:15Z DBG msg='499 Client Closed Request' caused by: context canceled
{"ClientAddr":"10.0.0.3:60762","ClientHost":"10.0.0.3","ClientPort":"60762","ClientUsername":"-","DownstreamContentSize":21,"DownstreamStatus":499,"Duration":1358074,"OriginContentSize":21,"OriginDuration":1307128,"OriginStatus":499,"Overhead":50946,"RequestAddr":"minio2.xxx.x","RequestContentSize":0,"RequestCount":79,"RequestHost":"minio2.xxx.x","RequestMethod":"PUT","RequestPath":"/agile-service/1/5724918fde504e3184ef3f2d161a3d8a@MYSQL.sql","RequestPort":"-","RequestProtocol":"HTTP/1.1","RequestScheme":"http","RetryAttempts":0,"RouterName":"minio@docker","ServiceAddr":"10.0.1.123:9000","ServiceName":"minio@docker","ServiceURL":{"Scheme":"http","Opaque":"","User":null,"Host":"10.0.1.123:9000","Path":"","RawPath":"","OmitHost":false,"ForceQuery":false,"RawQuery":"","Fragment":"","RawFragment":""},"StartLocal":"2024-06-05T01:16:15.144298495Z","StartUTC":"2024-06-05T01:16:15.144298495Z","entryPointName":"web","level":"info","msg":"","time":"2024-06-05T01:16:15Z"}
2024-06-05T01:16:16Z DBG msg='499 Client Closed Request' caused by: context canceled
{"ClientAddr":"10.0.0.3:60772","ClientHost":"10.0.0.3","ClientPort":"60772","ClientUsername":"-","DownstreamContentSize":21,"DownstreamStatus":499,"Duration":1784155,"OriginContentSize":21,"OriginDuration":1700142,"OriginStatus":499,"Overhead":84013,"RequestAddr":"minio2.xxx.x","RequestContentSize":0,"RequestCount":80,"RequestHost":"minio2.xxx.x","RequestMethod":"PUT","RequestPath":"/agile-service/1/5724918fde504e3184ef3f2d161a3d8a@MYSQL.sql","RequestPort":"-","RequestProtocol":"HTTP/1.1","RequestScheme":"http","RetryAttempts":0,"RouterName":"minio@docker","ServiceAddr":"10.0.1.123:9000","ServiceName":"minio@docker","ServiceURL":{"Scheme":"http","Opaque":"","User":null,"Host":"10.0.1.123:9000","Path":"","RawPath":"","OmitHost":false,"ForceQuery":false,"RawQuery":"","Fragment":"","RawFragment":""},"StartLocal":"2024-06-05T01:16:16.046793427Z","StartUTC":"2024-06-05T01:16:16.046793427Z","entryPointName":"web","level":"info","msg":"","time":"2024-06-05T01:16:16Z"}
2024-06-05T01:16:16Z DBG msg='499 Client Closed Request' caused by: context canceled
{"ClientAddr":"10.0.0.3:60786","ClientHost":"10.0.0.3","ClientPort":"60786","ClientUsername":"-","DownstreamContentSize":21,"DownstreamStatus":499,"Duration":1097654,"OriginContentSize":21,"OriginDuration":1035938,"OriginStatus":499,"Overhead":61716,"RequestAddr":"minio2.xxx.x","RequestContentSize":0,"RequestCount":81,"RequestHost":"minio2.xxx.x","RequestMethod":"PUT","RequestPath":"/agile-service/1/5724918fde504e3184ef3f2d161a3d8a@MYSQL.sql","RequestPort":"-","RequestProtocol":"HTTP/1.1","RequestScheme":"http","RetryAttempts":0,"RouterName":"minio@docker","ServiceAddr":"10.0.1.123:9000","ServiceName":"minio@docker","ServiceURL":{"Scheme":"http","Opaque":"","User":null,"Host":"10.0.1.123:9000","Path":"","RawPath":"","OmitHost":false,"ForceQuery":false,"RawQuery":"","Fragment":"","RawFragment":""},"StartLocal":"2024-06-05T01:16:16.179774176Z","StartUTC":"2024-06-05T01:16:16.179774176Z","entryPointName":"web","level":"info","msg":"","time":"2024-06-05T01:16:16Z"}
2024-06-05T01:16:16Z DBG msg='499 Client Closed Request' caused by: context canceled
{"ClientAddr":"10.0.0.3:60500","ClientHost":"10.0.0.3","ClientPort":"60500","ClientUsername":"-","DownstreamContentSize":21,"DownstreamStatus":499,"Duration":965192,"OriginContentSize":21,"OriginDuration":903736,"OriginStatus":499,"Overhead":61456,"RequestAddr":"minio2.xxx.x","RequestContentSize":0,"RequestCount":82,"RequestHost":"minio2.xxx.x","RequestMethod":"PUT","RequestPath":"/agile-service/1/5724918fde504e3184ef3f2d161a3d8a@MYSQL.sql","RequestPort":"-","RequestProtocol":"HTTP/1.1","RequestScheme":"http","RetryAttempts":0,"RouterName":"minio@docker","ServiceAddr":"10.0.1.123:9000","ServiceName":"minio@docker","ServiceURL":{"Scheme":"http","Opaque":"","User":null,"Host":"10.0.1.123:9000","Path":"","RawPath":"","OmitHost":false,"ForceQuery":false,"RawQuery":"","Fragment":"","RawFragment":""},"StartLocal":"2024-06-05T01:16:16.959618027Z","StartUTC":"2024-06-05T01:16:16.959618027Z","entryPointName":"web","level":"info","msg":"","time":"2024-06-05T01:16:16Z"}
2024-06-05T01:16:17Z DBG msg='499 Client Closed Request' caused by: context canceled
{"ClientAddr":"10.0.0.3:60506","ClientHost":"10.0.0.3","ClientPort":"60506","ClientUsername":"-","DownstreamContentSize":21,"DownstreamStatus":499,"Duration":1033627,"OriginContentSize":21,"OriginDuration":971332,"OriginStatus":499,"Overhead":62295,"RequestAddr":"minio2.xxx.x","RequestContentSize":0,"RequestCount":83,"RequestHost":"minio2.xxx.x","RequestMethod":"PUT","RequestPath":"/agile-service/1/5724918fde504e3184ef3f2d161a3d8a@MYSQL.sql","RequestPort":"-","RequestProtocol":"HTTP/1.1","RequestScheme":"http","RetryAttempts":0,"RouterName":"minio@docker","ServiceAddr":"10.0.1.123:9000","ServiceName":"minio@docker","ServiceURL":{"Scheme":"http","Opaque":"","User":null,"Host":"10.0.1.123:9000","Path":"","RawPath":"","OmitHost":false,"ForceQuery":false,"RawQuery":"","Fragment":"","RawFragment":""},"StartLocal":"2024-06-05T01:16:17.1880929Z","StartUTC":"2024-06-05T01:16:17.1880929Z","entryPointName":"web","level":"info","msg":"","time":"2024-06-05T01:16:17Z"}