MediaToolkit

Web-ready standardized file processing and serialization

Features

Read, load and convert to standard file types with a common interface. Especially useful for code that works with multiple file types like images, audio, video, etc.

Load and convert from and to common data types: - numpy arrays - file paths - bytes - base64 - json - urls - etc.

Transmit files between services with a common interface - Native FastSDK and FastTaskAPI integration - Supports httpx, requests

Work with native python libs like BytesIO.

Only use the file types you need, no unnecessary dependencies.

Installation

You can install the package with PIP, or clone the repository.

```bash

install from pypi

pip install media-toolkit

install without dependencies: this is useful if you only need the basic functionality (working with files)

pip install media-toolkit --no-deps

if you want to use certain file types, and convenience functions

pip install media-toolkit[VideoFile] # or [AudioFile, VideoFile, ...]

install from github for newest release

pip install git+git://github.com/SocAIty/media-toolkit ```

The package checks if you have missing dependencies for certain file types while using. Use the --no-deps flag for a minimal tiny pure python installation. The package with dependencies is quite small < 39kb itself.

Note: for VideoFile you will also need to install ffmpeg

Usage

Create a media-file from any data type

The library automatically detects the data type and loads it correctly.

```python from media_toolkit import MediaFile, ImageFile, AudioFile, VideoFile

could be a path, url, base64, bytesio, file_handle, numpy array ...

arbitrary_data = "...."

Instantiate an image file

newfile = ImageFile().fromany(arbitrary_data) ```

All files (ImageFile, AudioFile, VideoFile) types support the same interface / methods.

Explicitly load from a certain type.

This method is more secure than fromany, because it definitely uses the correct method to load the file. ```python newfile = MediaFile()

newfile.fromfile("path/to/file") newfile.fromfile(open("path/to/file", "rb")) newfile.fromnumpyarray(myarray) newfile.frombytes(b'bytes') newfile.frombase64('base64string') newfile.fromstarletteuploadfile(starletteuploadfile)

```

Convert to any format or write to file

Supports common serialization methods like bytes(), np.array(), dict()

```python myfile = ImageFile().fromfile("path/to/my_image.png")

myfile.save("path/to/newfile.png")
asnumpyarray = myfile.tonumpyarray() asnumpyarray = np.array(myfile)

asbytes = myfile.tobytes() asbytes = bytes(myfile) asbase64 = myfile.tobase64() asjson = myfile.to_json() ```

Working with Collections of Files

MediaList

A flexible list that can handle multiple media files with type safety:

```python from media_toolkit import MediaList, AudioFile

Create a list that only accepts AudioFiles

audio_list = MediaListAudioFile

Add files to the list

audiolist.append("path/to/audio.mp3") audiolist.extend(["url1", "url2"])

Process all files

for audio in audiolist: print(audio.filesize())

Convert all files to base64

base64files = audiolist.to_base64() ```

MediaDict

A dictionary for organizing media files with keys:

```python from media_toolkit import MediaDict, ImageFile

Create a dictionary that only accepts ImageFiles

image_dict = MediaDictImageFile

Add files with keys

imagedict["profile"] = "path/to/profile.jpg" imagedict["banner"] = "https://example.com/banner.png"

Process files

for key, image in imagedict.items(): print(f"{key}: {image.filesize()}")

Convert to JSON

jsondata = imagedict.to_json() ```

Both MediaList and MediaDict support: - Type safety with generic types (e.g., MediaList[AudioFile]) - Lazy loading of files - Batch processing - Common operations (tobase64, tobytes, etc.) - Nested structures (MediaDict inside MediaList and vice versa)

Working with VideoFiles

The VideoFiles wrap the famous vidgear package as well as pydub. VideoFiles support extra methods like audio extraction, combining video and audio. Vidgear is a powerful video processing library that supports many video formats and codecs and is known for fast video processing.

```python

load the video file

vf = VideoFile().fromfile("testfiles/testvid1.mp4")

extract audio_file

vf.extractaudio("extractedaudio.mp3")

stream the video

for img, audio in vf.tovideostream(include_audio=True): cv2.imwrite("outtest.png", img)

add audio to an videofile (supports files and numpy.array)

vf.add_audio("path/to/audio.mp3")

create a video from a folder

VideoFile().fromdir("path/to/imagefolder", audio=f"extractedaudio.mp3", framerate=30)

create a video from a video stream

fromstream = VideoFile().fromvideostream(vf.tovideostream(include_audio=True)) ```

Web-features

We intent to make transmitting files between services as easy as possible. Here are some examples for services and clients.

FastTaskAPI - Services

The library supports the FastTaskAPI and FastSDK for easy file transmission between services. Simply use the files in the taskendpoint function definition and transmitted data will be converted. Check out the FastTaskAPI documentation for more information. ```python from fasttask_api import ImageFile, AudioFile, VideoFile

@app.taskendpoint("/myfileupload") def myuploadimage(image: ImageFile, audio: AudioFile, video: VideoFile): imageasnparray = np.array(image) ```

fastAPI - services

You can use the files in fastapi and transform the starlette upload file to a MediaFile. python @app.post("/upload") async def upload_file(file: UploadFile = File(...)): mf = ImageFile().from_any(file) return {"filename": file.filename}

Client with: requests, httpx

To send a MediaFile to an openapi endpoint you can use the following method:

```python import httpx

mymediafile = ImageFile().fromfile("path/to/myimage.png") myfiles = { "paramname": mymediafile.tohttpxsendabletuple() ... } response = httpx.Client().post(url, files=my_files) ```

How it works

If media-file is instantiated with from_* it converts it to an intermediate representation. The to_* methods then convert it to the desired format.

Currently the intermediate representation is supported in memory with (BytesIO) or on disk with temporary files.

ToDo:

• [x] decreasing redundancies for fileinfo() method