Google Cloud Bucket: create explicit dirs

gsmkdirs.sh

#!/bin/bash

##
## Creates explicit dirs on the bucket
##

function print_help() {
  echo "Usage:" $(basename $0) bucket_mounted_dir
  exit 1
}

if [[ -z $1 || ! -d $1 ]]; then
  print_help
fi

# Expect mount dir is /*/*
path=$(echo $1|sed -rn 's|^/[^/]+/[^/]+/?||p')
mnt=${1%$path}
mnt=${mnt%/}
path=${path%/}
bucket=$(mount | grep -F "$mnt")
bucket=${bucket%% *}
if [[ -z "$bucket" || "$bucket" == /* ]]; then
  echo No or wrong mount point for $mnt
  exit 1
fi
bucket=gs://"$bucket"

echo Creating dirs for $bucket/$path in $mnt/$path ...
echo Press ENTER to continue ...
read

declare -A cdirs # created dirs

gsutil ls -r "$bucket/$path" | while IFS='' read -r line; do
  if [[ -z "$line" || "$line" == */ || "$line" == *: ]]; then continue; fi
  #echo Line $line
  dir=${line#$bucket}
  if [[ -z $dir ]]; then continue; fi
  dir=${dir%/*} # dirname
  #echo Examining $dir
  if [[ -z $dir || ${cdirs[$dir]+1} ]]; then continue; fi
  cdirs[$dir]=1
  diskdir="$mnt$dir"
  if [[ -e "$diskdir" ]]; then
    echo Dir already exists $diskdir
    continue
  fi
  echo Creating $diskdir
  mkdir -p "$diskdir"
done

Hi Ondrej, can you please explain how this script should be used? Should it be run in the directory where the bucket is mounted?

thank you very much, your code is a life saver.

Hi Ondrej, can you please explain how this script should be used? Should it be run in the directory where the bucket is mounted?
you can run this code anywhere, it comes with help how to use it.

@yalaudah The script needs a mounted by gcsfuse and working gsutil command. The mounted gcsfuse dir, where you want to created dirs is a first argument of the script. (The gs:// uri for gsutil is recognized from the output of mount command.)