Recursively upload a folder with Python

Learn how to upload a local folder tree to Box with a recursive Python script that creates folders, uploads files, and handles naming conflicts.

This tutorial shows you how to build a recursive upload script that combines , , and to handle duplicate files, duplicate folders, and large files automatically.

The code samples in this tutorial use the .

Prerequisites

• A Box application with the scope (root_readwrite) enabled
• Python 3.8 or later installed on your computer
• The Box Python SDK installed (pip install "boxsdk~=10")
• An authenticated
• Completion of the tutorial, which defines the file_upload and file_upload_chunked helper functions this tutorial reuses to avoid duplicating upload logic

Handle duplicate folders

Before you access a subfolder, check whether it already exists in Box. If a folder with the same name exists, use the existing folder instead of creating a new one:

Python v10

from box_sdk_gen import BoxAPIError, CreateFolderParent


def create_or_get_box_folder(client, folder_name, parent_folder_id):
    try:
        folder = client.folders.create_folder(
            folder_name, CreateFolderParent(id=parent_folder_id)
        )
    except BoxAPIError as err:
        if err.response_info.body.get("code") == "item_name_in_use":
            folder_id = err.response_info.context_info["conflicts"][0][
                "id"
            ]
            folder = client.folders.get_folder_by_id(folder_id)
        else:
            raise
    return folder

Recursively upload a folder

Combine the helper functions into a single recursive function that walks the local directory tree. For each item, the function:

1. Creates the folder in Box, or retrieves the existing folder, and then recurses into it.
2. Checks the file size to choose a direct or chunked upload. Files that are 20 MB or larger use chunked upload; smaller files use direct upload.
3. Uploads a new version when a file with the same name already exists, by using the helpers from the upload methods tutorial.

Python v10

import pathlib

MIN_CHUNKED_SIZE = 20 * 1024 * 1024  # 20 MB


def folder_upload(
    client,
    box_base_folder_id,
    local_folder_path,
    min_chunked_size=MIN_CHUNKED_SIZE,
):
    local_folder = pathlib.Path(local_folder_path)
    for item in local_folder.iterdir():
        if item.is_dir():
            new_box_folder = create_box_folder(
                client, item.name, box_base_folder_id
            )
            folder_upload(client, new_box_folder.id, item, min_chunked_size)
        else:
            if item.stat().st_size < min_chunked_size:
                file_upload(client, str(item), box_base_folder_id)
            else:
                file_upload_chunked(client, str(item), box_base_folder_id)

The min_chunked_size parameter defaults to 20 MB, which is the minimum size for . You can increase this threshold, but you cannot decrease it.

Complete example

The following script authenticates a Box client, creates a destination folder (or reuses an existing one), and recursively uploads the contents of a local directory. It reuses the create_box_folder helper to get or create the destination folder in the root folder ("0"):

Python v10

def main():
    client = get_authenticated_client()
    demo_folder = create_box_folder(client, "my-upload-folder", "0")
    folder_upload(client, demo_folder.id, "/path/to/local/folder")


if __name__ == "__main__":
    main()

Resources