Downloading segmented video from vimeo

vimeo-download.py

import requests
import base64
from tqdm import tqdm


master_json_url = 'https://178skyfiregce-a.akamaihd.net/exp=1474107106~acl=%2F142089577%2F%2A~hmac=0d9becc441fc5385462d53bf59cf019c0184690862f49b414e9a2f1c5bafbe0d/142089577/video/426274424,426274425,426274423,426274422/master.json?base64_init=1'
base_url = master_json_url[:master_json_url.rfind('/', 0, -26) + 1]

resp = requests.get(master_json_url)
content = resp.json()

heights = [(i, d['height']) for (i, d) in enumerate(content['video'])]
idx, _ = max(heights, key=lambda (_, h): h)
video = content['video'][idx]
video_base_url = base_url + video['base_url']
print 'base url:', video_base_url

filename = 'video_%d.mp4' % video['id']
print 'saving to %s' % filename

video_file = open(filename, 'wb')

init_segment = base64.b64decode(video['init_segment'])
video_file.write(init_segment)

for segment in tqdm(video['segments']):
    segment_url = video_base_url + segment['url']
    resp = requests.get(segment_url, stream=True)
    if resp.status_code != 200:
        print 'not 200!'
        print resp
        print segment_url
        break
    for chunk in resp:
        video_file.write(chunk)

video_file.flush()
video_file.close()

so I skidded modifications from comments and added a bit of multi threading. It downloads video and audio, accepts playlist.json and master.json (mainly for playlist.json).

import os
import sys
import base64
import requests
import subprocess
from concurrent.futures import ThreadPoolExecutor

from tqdm import tqdm
from moviepy.editor import *
import ffmpeg


url = input('enter [master|playlist].json url: ')
name = input('enter output name: ')

if 'master.json' in url:
    url = url[:url.find('?')] + '?query_string_ranges=1'
    url = url.replace('master.json', 'master.mpd')
    print(url)
    subprocess.run(['youtube-dl', url, '-o', name])
    sys.exit(0)


def download_segment(segment_url, segment_path):
    resp = requests.get(segment_url, stream=True)
    if resp.status_code != 200:
        print('not 200!')
        print(segment_url)
        return
    with open(segment_path, 'wb') as segment_file:
        for chunk in resp:
            segment_file.write(chunk)

def download(what, to, base):
    print('saving', what['mime_type'], 'to', to)
    init_segment = base64.b64decode(what['init_segment'])
    
    segment_urls = [base + segment['url'] for segment in what['segments']]
    segment_paths = [f"segment_{i}.tmp" for i in range(len(segment_urls))]

    with ThreadPoolExecutor(max_workers=15) as executor:
        list(tqdm(executor.map(download_segment, segment_urls, segment_paths), total=len(segment_urls)))

    with open(to, 'wb') as file:
        file.write(init_segment)
        for segment_path in segment_paths:
            with open(segment_path, 'rb') as segment_file:
                file.write(segment_file.read())
            os.remove(segment_path)
    
    print('done')


name += '.mp4'
base_url = url[:url.rfind('/', 0, -26) + 1]
content = requests.get(url).json()

vid_heights = [(i, d['height']) for (i, d) in enumerate(content['video'])]
vid_idx, _ = max(vid_heights, key=lambda _h: _h[1])

audio_quality = [(i, d['bitrate']) for (i, d) in enumerate(content['audio'])]
audio_idx, _ = max(audio_quality, key=lambda _h: _h[1])

video = content['video'][vid_idx]
audio = content['audio'][audio_idx]
base_url = base_url + content['base_url']

video_tmp_file = 'video.mp4'
audio_tmp_file = 'audio.mp4'

download(video, video_tmp_file, base_url + video['base_url'])
download(audio, audio_tmp_file, base_url + audio['base_url'])

def combine_video_audio(video_file, audio_file, output_file):
    try:
        video_stream = ffmpeg.input(video_file)
        audio_stream = ffmpeg.input(audio_file)
        
        ffmpeg.output(video_stream, audio_stream, output_file, vcodec='copy', acodec='copy').run(overwrite_output=True)
        
        print(f"Archivo combinado guardado como {output_file}")
    except ffmpeg.Error as e:
        print(f"Error al combinar archivos: {e.stderr.decode()}")

combine_video_audio('video.mp4', 'audio.mp4', name)

os.remove(video_tmp_file)
os.remove(audio_tmp_file)

so I skidded modifications from comments and added a bit of multi threading. It downloads video and audio, accepts playlist.json and master.json (mainly for playlist.json).

This definitely rocks!
IMHO it is worth a dedicated docker image such as
Dockerfile

FROM python
RUN apt-get update && apt-get install -y \
    youtube-dl \
    && ln -s /usr/bin/yt-dlp /usr/local/bin/youtube-dl \
    && rm -rf /var/lib/apt/lists/*
RUN python -m pip install requests tqdm moviepy ffmpeg-python
COPY video.py /video.py
VOLUME /downloads
WORKDIR /downloads
ENTRYPOINT ["python"]
CMD ["/video.py"]

possibly checking also for env vars, hence asking user input just as a fallback
video.py

...
url = os.getenv("SRC_URL") or input('enter [master|playlist].json url: ')
name = os.getenv("OUT_FILE") or input('enter output name: ')
...

so that it is portable and launched by

docker build -t vimeo-dl .
docker run \
  -e 'SRC_URL=https://...' \
  -e 'OUT_FILE=/downloads/video.mp4' \
  -v $(pwd)/out:/downloads \
  --rm -it vimeo-dl

or with docker compose
compose.yml

services:
  downloader:
    build:
      context: .
    volumes:
    - ./out:/downloads
    environment:
    - SRC_URL=${SRC_URL}
    - OUT_FILE=${OUT_FILE}

.env

SRC_URL=https://...
OUT_FILE=/downloads/video.mp4

I also made a portable version (for windows) with some improvements, now it asks the number of workers in case the computer or network is not so powerful (default 5 and max 15, we do not want vimeo to notice someone is opening 30 connections to their server lol)

https://mega.nz/file/ZNwjkSqY#FQ4bgiQiOKt_EpLSMnyu1uSZ-7t6WFqHmjuwWxysVjw

It is just the .py compiled to exe with this tool: auto_py_to_exe
It includes the ffmpeg.exe, since the ffmpeg library still needs the executable
(I did not include youtube-dl, it is used for master.json but my script was mainly created for Playlist. If you want to add youtube-dl, you can see in the code it should be in the root folder)

I also made a portable version (for windows) with some improvements, now it asks the number of workers

Great! I could contribute the linux docker image based on your script, i.e. on a dedicated GH repo.

Sure!

Here you go: https://github.com/davidecavestro/vimeo-dl
I consider it a mere starting point, so if you or anyone else wants to fork and maintain it, I'll be happy to pass it along.

so I skidded modifications from comments and added a bit of multi threading. It downloads video and audio, accepts playlist.json and master.json (mainly for playlist.json).

import os
import sys
import base64
import requests
import subprocess
from concurrent.futures import ThreadPoolExecutor

from tqdm import tqdm
from moviepy.editor import *
import ffmpeg


url = input('enter [master|playlist].json url: ')
name = input('enter output name: ')

if 'master.json' in url:
    url = url[:url.find('?')] + '?query_string_ranges=1'
    url = url.replace('master.json', 'master.mpd')
    print(url)
    subprocess.run(['youtube-dl', url, '-o', name])
    sys.exit(0)


def download_segment(segment_url, segment_path):
    resp = requests.get(segment_url, stream=True)
    if resp.status_code != 200:
        print('not 200!')
        print(segment_url)
        return
    with open(segment_path, 'wb') as segment_file:
        for chunk in resp:
            segment_file.write(chunk)

def download(what, to, base):
    print('saving', what['mime_type'], 'to', to)
    init_segment = base64.b64decode(what['init_segment'])
    
    segment_urls = [base + segment['url'] for segment in what['segments']]
    segment_paths = [f"segment_{i}.tmp" for i in range(len(segment_urls))]

    with ThreadPoolExecutor(max_workers=15) as executor:
        list(tqdm(executor.map(download_segment, segment_urls, segment_paths), total=len(segment_urls)))

    with open(to, 'wb') as file:
        file.write(init_segment)
        for segment_path in segment_paths:
            with open(segment_path, 'rb') as segment_file:
                file.write(segment_file.read())
            os.remove(segment_path)
    
    print('done')


name += '.mp4'
base_url = url[:url.rfind('/', 0, -26) + 1]
content = requests.get(url).json()

vid_heights = [(i, d['height']) for (i, d) in enumerate(content['video'])]
vid_idx, _ = max(vid_heights, key=lambda _h: _h[1])

audio_quality = [(i, d['bitrate']) for (i, d) in enumerate(content['audio'])]
audio_idx, _ = max(audio_quality, key=lambda _h: _h[1])

video = content['video'][vid_idx]
audio = content['audio'][audio_idx]
base_url = base_url + content['base_url']

video_tmp_file = 'video.mp4'
audio_tmp_file = 'audio.mp4'

download(video, video_tmp_file, base_url + video['base_url'])
download(audio, audio_tmp_file, base_url + audio['base_url'])

def combine_video_audio(video_file, audio_file, output_file):
    try:
        video_stream = ffmpeg.input(video_file)
        audio_stream = ffmpeg.input(audio_file)
        
        ffmpeg.output(video_stream, audio_stream, output_file, vcodec='copy', acodec='copy').run(overwrite_output=True)
        
        print(f"Archivo combinado guardado como {output_file}")
    except ffmpeg.Error as e:
        print(f"Error al combinar archivos: {e.stderr.decode()}")

combine_video_audio('video.mp4', 'audio.mp4', name)

os.remove(video_tmp_file)
os.remove(audio_tmp_file)

I'm a complete noob. I only know how to download vimeo videos with yt-dlp in the way that I open cmd and type

yt-dlp --format options--

But recently, that didn't work anymore for quite a few videos. I also found the playlist.json link, but I don't know at all how to install python, let alone how to use this script. Would you be so kind to explain me in detail how to install and use this (what I have to type in cmd), maybe with an example? Would be greatly appreciated. Thanks

You can use the portable version here

But if you still want to use the script, here are the steps:

1. To install python, type "python" in CMD, if it is not installed, it will get you install it. If it was installed, CMD will change to a ">>" symbol, you can close it.
2. The script needs youtube-dl for master.json and ffmpeg for playlist.json. (Idk how youtube-dl works, I guess it can be installed and doesn't matter where do you put it, but ffmpeg.exe needs to be in the same folder as the script. Youtube-dl should have a ffmpeg.exe, but If not, you can find it in the portable version of my script)
3. run the script. With python installed, you can open a cmd in the script folder and just type "python filename.py".

Note: the portable version does not have youtube-dl, as I said, I was mainly focused in playlist.json

I guess docker can be installed on windows so you may be able to use davidecavestro's portable version too: https://github.com/davidecavestro/vimeo-dl

Regarding the portable version, a virus is detected. Windows marks it as "Trojan:Script/Wacatac.B!ml"! Why's that?

Regarding your point 2.: I already got yt-dlp and ffmpeg. Does the script work with yt-dlp as well, or only youtube-dl?
Do I still need to run the command "python -m pip install requests tqdm moviepy" to install those packages?

Yes, for running the script you need to install it's packages, and nope, I think it needs youtube-dl since the subprocess command is "youtube-dl".

About the flag from defender, I don't really know, it's the same script just compiled with auto-py-to-exe:
https://pypi.org/project/auto-py-to-exe/2.5.0/

Nothing malicious from my side, and I don't think pypi.org would allow any malicious package, so I would say it is a false positive (defender is a bit stressful sometimes, that's why I passed to kaspersky free which is still 4 times better)

Yes, for running the script you need to install it's packages, and nope, I think it needs youtube-dl since the subprocess command is "youtube-dl".

About the flag from defender, I don't really know, it's the same script just compiled with auto-py-to-exe: https://pypi.org/project/auto-py-to-exe/2.5.0/

Nothing malicious from my side, and I don't think pypi.org would allow any malicious package, so I would say it is a false positive (defender is a bit stressful sometimes, that's why I passed to kaspersky free which is still 4 times better)

I installed all the packages (requests, tqdm, moviepy) and saved the code of the script to vimeo_script.py. When I try to run it with python vimeo_script.py, I get the following Error:

Traceback (most recent call last):
  File "C:\Users\Nico\Desktop\Vimeo Videos\vimeo_script.py", line 9, in <module>
    from moviepy.editor import *
ModuleNotFoundError: No module named 'moviepy.editor'

I deinstalled moviepy and installed it again, but no change. I also tested it in a virtual environment, no change. Can you help me with that? Thanks in advance!

If it gives problems try to remove the moviepy import.
I think I'm not actually using it anywhere

moviepy.editor suppression and simplified importation
Before v2.0, it was advised to import from moviepy.editor whenever you needed to do some sort of manual operations, such as previewing or hand editing, because the editor package handled a lot of magic and initialization, making your life easier, at the cost of initializing some complex modules like pygame.

With version 2.0, the moviepy.editor namespace simply no longer exists. You simply import everything from moviepy like this:

from moviepy import * # Simple and nice, the all is set in moviepy so only useful things will be loaded
from moviepy import VideoFileClip # You can also import only the things you really need

https://zulko.github.io/moviepy/getting_started/updating_to_v2.html