Large File Uploads

Status Quo and Background

To meet growing business needs and overcome the current 10MB file size limit in our file upload specifications, the TTS Partner Team is officially launching a large file upload solution. The previous solution loaded entire files into memory, posing a potential Out of Memory (OOM) risk to the gateway service and limiting the expansion of business scenarios.

The new solution aims to provide stable and efficient large file upload capabilities, supporting single files up to 1GB to accommodate the demand for uploading videos, high-definition images, and other large files.

• The Customer Service scenario supports a maximum file size of 100 MB, while the Shoppable Video scenario supports a maximum file size of 500 MB.

Technical Solution Overview

This solution adopts a "business file gateway + chunked upload" approach to support large file transfers. The core of this strategy is to slice large files on the client side, upload the chunks one by one to a dedicated file gateway, and finally have the gateway merge them on the server side.

• Direct Upload for Small Files: For files smaller than 10MB, developers can continue to use the existing direct upload method without chunking, simplifying the process for small files.
• Chunked Upload: For files larger than 10MB, the chunked upload model provided in this solution must be used.

To ensure a stable and secure upload process, the solution includes several key capabilities:

• Upload Token: The upload_token obtained from the upload initialization API is the sole credential for subsequent chunk upload operations. It is time-limited to ensure each upload session is independent and secure.
• Security and Authentication: The solution inherits the universal authentication system of the API gateway and includes customized, scenario-specific authentication for file uploads to support calls from different business parties. Additionally, user-level rate and volume limiting mechanisms are in place to prevent malicious uploads.
• Compliance: The file gateway and API gateway share the same domain name and reuse existing traffic scheduling policies, ensuring that data is transmitted within the corresponding compliance unit.

How to Integrate

Large File Upload Process Details

Large file uploads are divided into three main steps: 1. Initialize Upload -> 2. Chunk Upload and Completion -> 3. Bind Business Resource.

ID	Step	Flow	API Path	Description
1	Initialize Upload		[POST] /open/:version/file_upload/init	Before uploading any file chunks, you must call the initialization API to inform the server of the file's basic information and obtain the upload_url and upload_token required for subsequent operations.
2	Chunk Upload and Completion		[PUT] <uploader_url> (returned by the init API)	After obtaining the upload_url and upload_token, you need to split the file into multiple chunks on the client side and number each chunk (chunk_num, starting from 1). Then, upload each chunk to the upload_url by repeatedly calling the PUT request.
Important Note: The gateway handles file merging automatically. After the last chunk is successfully uploaded, the gateway will trigger the merge operation and return the final resource_id directly in the response for the last chunk's upload. No extra API call is needed to notify the server to merge the file.
3	Bind Resource		For example, in the Customer Service business, the resource binding API is
[POST]/customer_service/:version/conversations/{conversation_id}/messages	After obtaining the resource_id, you need to call the business-specific "resource binding" API to associate this resource_id with a business entity (e.g., a product, an article), making it usable in your business context.
In the Customer Service business, the resource binding API is Send Message, and the resource_id needs to be passed as the vid parameter.

API Details

1. Initialize Upload

The Upload File Init API is used to obtain the target URL and authentication token required for chunked uploads.

• Endpoint: [POST] /open/:version/file/init
• Common Query Parameters:

access_token: {Your access token} app_key: {Your app key} timestamp: {Current timestamp} sign: {Request signature} Common Query Parameters

: - access_token: {Your access token}

• app_key: {Your app key}

• timestamp: {Current timestamp}

• sign: {Request signature}

• Request Body:

Parameter	Type	Description
file_size	Integer	Total file size in bytes.
file_name	String	File name, including the extension (e.g., "my_video.mp4").
total_chunk_count	Integer	Total number of chunks.
file_type	String	File type. Enum values: video, image, pdf.
target_path	String	The API path of the target resource binding endpoint, used for permission checks and routing. For example: [POST]/customer_service/:version/conversations/{conversation_id}/messages.

• Success Response:

JSONWord Wrap

{
  "upload_url": "{uploader_url}",
  "upload_token": "{your_unique_upload_token}"
}

2. Upload File Chunk

This API is used to upload a single file chunk. We require that a single chunk must not be smaller than 5MB and not larger than 30MB.

• Endpoint: [PUT] <uploader_url> (returned by the init API)
• Query Parameters:

Parameter	Type	Description
upload_token	String	The upload token obtained from the initialization API.
chunk_num	Integer	The sequence number of the current chunk, starting from 1.
app_key	string	Your AppKey.
shop_cipher	String	Use this property to pass shop information in requesting the API. Failure in passing the correct value when requesting the API for cross-border shops will return incorrect response.
Get by API Get Authorization Shop
This field is required when the authorization user_type = 0 (Seller) or user_type = 3(Partner) and will be ignored when the authorization user_type = 1 (Creator).

• Headers:

Header	Type	Description
Content-Type	String	The MIME type of the current chunk. Must be compatible with the file_type declared during initialization.
x-tts-access-token	String	The user's access token for authentication.

• Request Body:

(binary): The binary data of the file. Request Body

: - (binary): The binary data of the file.

• Response:

Chunk upload successful: Returns part_id. All chunks uploaded: After the last chunk is successfully uploaded, returns the final resource_id. Response

: - Chunk upload successful: Returns part_id.

• All chunks uploaded: After the last chunk is successfully uploaded, returns the final resource_id.

JSONWord Wrap

{
  "resource_id": "{final_resource_id}"
}

• Supported Content-Types:

File Category	Supported Content-Types
Video (video)	video/mp4, video/quicktime, video/webm
Image (image)	image/png, image/jpeg

3. Bind Resource

Used to associate the successfully uploaded file resource with your business logic.

• Endpoint: Varies by business. For example, Customer Service uses: [POST]/customer_service/:version/conversations/{conversation_id}/messages.
• Request/Response: You need to pass the resource_id obtained in the previous step as a parameter.

Parameters and Validation Process

During the chunked upload process, the gateway performs strict validation:

1. Size Accumulation Check: Each time a chunk is uploaded, the gateway accumulates the total size of uploaded chunks and compares it against the file_size declared at initialization to ensure the total upload size does not exceed the expected amount.
2. Content-Type Check: The Content-Type of each chunk is validated to ensure it is consistent with the file_type declared at initialization.
3. Merge Trigger: When the gateway detects that all chunks (based on total_chunk_count) have been successfully uploaded, it automatically calls the underlying storage service's CommitUploadParts API to complete the file merge.

Code Examples

cURL Example

1. Initialize Upload

BASHWord Wrap

curl -X POST 'https://your-api-domain.com/open/202505/file/init?access_token=YOUR_ACCESS_TOKEN&app_key=YOUR_APP_KEY×tamp=...&sign=...' \  
-H 'Content-Type: application/json' \  
-d '{  
    "file_size": 20971520,  
    "file_name": "test_video.mp4",  
    "total_chunk_count": 1,  
    "file_type": "video",  
    "target_path": "[POST]/affiliate_creator/202505/videos"  
}'

1. Upload Chunk

BASHWord Wrap

# Assume the previous step returned upload_url and upload_token  
curl -X PUT 'https://returned-uploader-url.com/file/v1/upload?upload_token=RETURNED_UPLOAD_TOKEN&chunk_num=1' \  
-H 'Content-Type: video/mp4' \  
-H 'Content-Length: 20971520' \  
-H 'x-tts-access-token: USER_ACCESS_TOKEN' \  
--data-binary '@/path/to/your/local/file_chunk_1.dat'

Golang Example

The following example demonstrates how to read a local file, split it into chunks of a specified size, and upload them sequentially.

GOWord Wrap

package api_call  
  
import (  
        "bytes"  
        "context"  
        "crypto/hmac"  
        "crypto/sha256"  
        "encoding/hex"  
        "encoding/json"  
        "errors"  
        "log"  
        "mime"  
        "net/http"  
        "os"  
        "sort"  
        "strconv"  
        "time"  
)  
  
const chunkSize = 10 * 1024 * 1024  
const minTailSize = 5 * 1024 * 1024  
  
type OpenAPIClient struct {  
        BaseURL   string  
        Token     string  
        AppKey    string  
        AppSecret string  
        HTTP      *http.Client  
}  
  
func New(baseURL, token, appKey, appSecret string) *OpenAPIClient {  
        return &OpenAPIClient{BaseURL: baseURL, Token: token, AppKey: appKey, AppSecret: appSecret, HTTP: &http.Client{}}  
}  
  
type UploadInitRequest struct {  
        FileSize        int64  `json:"file_size"`  
        TotalChunkCount int    `json:"total_chunk_count"`  
        FileType        string `json:"file_type"`  
        TargetPath      string `json:"target_path"`  
        FileName        string `json:"file_name"`  
}  
  
func (c *OpenAPIClient) UploadInit(ctx context.Context, body UploadInitRequest) (*http.Response, []byte, error) {  
        urlStr := c.BaseURL + "/open/202512/file/init"  
        headers := map[string]string{  
                "Content-Type": "application/json",  
        }  
        if c.Token != "" {  
                headers["x-tts-access-token"] = c.Token  
        }  
        buf := &bytes.Buffer{}  
        if err := json.NewEncoder(buf).Encode(body); err != nil {  
                return nil, nil, err  
        }  
        ts := strconv.FormatInt(time.Now().Unix(), 10)  
        path := "/open/202512/file/init"  
        params := map[string]string{  
                "timestamp": ts,  
                "app_key":   c.AppKey,  
        }  
        sign := c.computeSign(path, params, headers["Content-Type"], buf.Bytes())  
        query := map[string]string{  
                "timestamp": ts,  
                "sign":      sign,  
                "app_key":   c.AppKey,  
        }  
        resp, respBody, err := APICall(urlStr, http.MethodPost, headers, query, buf)  
        if err != nil {  
                log.Printf("UploadInit Error=%v", err)  
                return resp, respBody, err  
        }  
  
        // Print API call result logs  
        // Only print x-tt-logid from the headers  
        logid := resp.Header.Get("X-Tt-Logid")  
        log.Printf("UploadInit resp, statusCode %d, logid %s, body %s", resp.StatusCode, logid, respBody)  
  
        return resp, respBody, nil  
}  
  
func (c *OpenAPIClient) ChunkUpload(ctx context.Context, urlStr, uploadToken string, filePath string, totalChunkCount int) (*http.Response, []byte, error) {  
        headers := map[string]string{}  
        headers["Content-Type"] = "video/mp4"  
        if c.Token != "" {  
                headers["x-tts-access-token"] = c.Token  
        }  
        // Read a file into memory  
        data, err := os.ReadFile(filePath)  
        if err != nil {  
                return nil, nil, err  
        }  
        // Default chunk size is 20 MB; if the last chunk is smaller than 5 MB, it will be merged with the previous chunk.  
  
        // Generate chunk boundaries  
        type rng struct{ start, end int }  
        var chunks []rng  
        for start := 0; start < len(data); start += chunkSize {  
                end := start + chunkSize  
                if end > len(data) {  
                        end = len(data)  
                }  
                chunks = append(chunks, rng{start: start, end: end})  
        }  
        // Merge the last chunk (if it is smaller than 5 MB and a previous chunk exists)  
        if len(chunks) >= 2 {  
                last := chunks[len(chunks)-1]  
                if lastLen := last.end - last.start; lastLen > 0 && lastLen < minTailSize {  
                        chunks[len(chunks)-2].end = last.end  
                        chunks = chunks[:len(chunks)-1]  
                }  
        }  
  
        var lastResp *http.Response  
        var lastBody []byte  
        for i, rg := range chunks {  
                chunk := data[rg.start:rg.end]  
                query := map[string]string{  
                        "upload_token": uploadToken,  
                        "chunk_num":    strconv.Itoa(i + 1),  
                        "app_key":      c.AppKey,  
                }  
                resp, body, err := APICall(urlStr, http.MethodPut, headers, query, bytes.NewReader(chunk))  
                if err != nil {  
                        log.Printf("Chunk %d: Error=%v", i+1, err)  
                        return resp, body, err  
                }  
                // Print API call result logs  
                // Only print x-tt-logid from the headers  
                logid := resp.Header.Get("X-Tt-Logid")  
                log.Printf("Chunk %d: StatusCode=%d Logid=%s, body=%s", i+1, resp.StatusCode, logid, string(body))  
  
                lastResp, lastBody = resp, body  
        }  
  
        return lastResp, lastBody, nil  
}  
  
// UploadFile combines UploadInit and ChunkUpload:  
// 1) Read the file size and calculate the total number of chunks based on the 20 MB chunk rule  
// 2) Call UploadInit to obtain the upload_token  
// 3) Use the upload_token to call ChunkUpload for chunked upload  
func (c *OpenAPIClient) UploadFile(ctx context.Context, filePath string) (*http.Response, []byte, error) {  
        fi, err := os.Stat(filePath)  
        if err != nil {  
                return nil, nil, err  
        }  
        size := fi.Size()  
        chunkCount := calcChunkCount(size)  
        // Update the file information and chunk count in the request  
        initReq := UploadInitRequest{  
                FileSize:        size,  
                TotalChunkCount: chunkCount,  
                FileName:        fi.Name(),  
                FileType:        "video",  
                TargetPath:      "[POST]/affiliate_creator/202505/videos",  
        }  
  
        initResp, initBody, err := c.UploadInit(ctx, initReq)  
        if err != nil {  
                return initResp, initBody, err  
        }  
  
        // Parse the upload_token  
        var parsed struct {  
                Code int `json:"code"`  
                Data struct {  
                        UploadToken string `json:"upload_token"`  
                        UploadURL   string `json:"upload_url"`  
                } `json:"data"`  
                Message   string `json:"message"`  
                RequestID string `json:"request_id"`  
        }  
        if err := json.Unmarshal(initBody, &parsed); err != nil {  
                return initResp, initBody, err  
        }  
        if parsed.Code != 0 {  
                return initResp, initBody, errors.New(parsed.Message)  
        }  
        if parsed.Data.UploadToken == "" {  
                return initResp, initBody, errors.New("upload_token is empty")  
        }  
  
        return c.ChunkUpload(ctx, parsed.Data.UploadURL, parsed.Data.UploadToken, filePath, chunkCount)  
}  
  
// Calculate the number of chunks based on the fixed 20 MB chunk size and the rule for merging the last chunk.  
func calcChunkCount(size int64) int {  
  
        if size <= 0 {  
                return 1  
        }  
        n := int(size / chunkSize)  
        rem := size % chunkSize  
        if rem > 0 {  
                n++  
        }  
        if rem > 0 && rem < minTailSize && n > 1 {  
                n--  
        }  
        if n <= 0 {  
                n = 1  
        }  
        return n  
}  
  
func (c *OpenAPIClient) computeSign(path string, params map[string]string, contentType string, body []byte) string {  
        // Copy and clean parameters: exclude sign and access_token  
        cleaned := make(map[string]string, len(params))  
        for k, v := range params {  
                cleaned[k] = v  
        }  
        delete(cleaned, "sign")  
        delete(cleaned, "access_token")  
  
        // Sort the keys in alphabetical order  
        keys := make([]string, 0, len(cleaned))  
        for k := range cleaned {  
                keys = append(keys, k)  
        }  
        sort.Strings(keys)  
  
        // Concatenate the string: path + key + value (sorted), and append body if necessary  
        input := path  
        for _, k := range keys {  
                input += k + cleaned[k]  
        }  
  
        // If Content-Type is not multipart/form-data, append the body.  
        if mediaType, _, err := mime.ParseMediaType(contentType); err == nil {  
                if mediaType != "multipart/form-data" {  
                        input += string(body)  
                }  
        } else {  
                // If parsing fails, handle conservatively: append the body.  
                input += string(body)  
        }  
  
        // Wrap with the secret and generate HMAC-SHA256.  
        input = c.AppSecret + input + c.AppSecret  
        mac := hmac.New(sha256.New, []byte(c.AppSecret))  
        if _, err := mac.Write([]byte(input)); err != nil {  
                return ""  
        }  
        return hex.EncodeToString(mac.Sum(nil))  
}

FAQ

Q1: How should I set the chunk size?

A:​It is recommended to set the chunk size between 5–20MB, with a maximum of 30MB. This range provides a good balance between stability and performance and can effectively reduce the risk of upload failures or timeouts.

Q2: Why does my 68MB local file show as only 65MB after being uploaded to the video cloud?

A:​This is normal.Videos undergo some processing (such as encapsulation or compression) after being uploaded to the video cloud, so there may be a slight discrepancy in file size. This difference will not affect video publishing or subsequent posting processes, so you can proceed with confidence.

Q3: Are there any limitations in the sandbox environment?

A:​Yes.The QPH (Queries Per Hour) limit for sandbox apps is 10. This is a normal restriction intended for functional verification only and is not suitable for high-frequency request scenarios.

Q4: What should I do before downloading the SDK?

A:​Before downloading the SDK, be sure to perform the following steps:

1. Click Check for updates;
2. Update your development tools to the latest version;
3. Then, download the SDK.

Failure to update your tools may lead to SDK download failures or version mismatch issues.