Skip to content


7zip missing move argument script

I needed to archive historical files in order to free up much needed disk space. This included a lot of Excel files which compress very well and can save a lot of space.

I was going to use the standard zip utility provided by my CentOS installation, but my colleague suggested I try 7zip instead. And sure enough the compression rate was way better. In some cases even 2x compared to zip.

My goal was to compress the files in a specified directory individually in their original location while retaining the original directory structure and then removing the original file. As far as the removal is concerned, the zip utility has a handy –move argument, which “moves the file into the archive” and thus removes the original. To my surprise 7zip did not have that functionality. The latest alpha desktop version seems to have it, but not the command line version. This meant I had to implement it myself in a bash script. While I was at it, I wrote a little extra logic and success/error logging.

Here’s the script:

Features:

  • Confirmation with the absolute path for security
  • Arguments for controlling runtime and verbosity

Todo/Problems

  • Silent mode (no output) not working as far as the 7zip command is concerned

#!/bin/sh
#
# SCRIPT FOR COMPRESSING FILES AND REMOVING ORIGINALS RECURSIVELY WITH 7ZIP
# Written by Kurt Martinsen on 2013/05/29
#
# At the time of writing, there was no --move argument in 7zip, which would remove the original file after successful compressing.
# Most of this can be replaced when such an option is available.
#
# This script does the following: 1. compresses files recursively 2. tests the archives 3. removes the original files.
# Retains file permissions and ownership for the achive, but depending on 7zip, might not do the same inside the archive.
# DOES NOT COMPRESS AND REMOVE COMPLETE DIRECTORIES, just the files inside them one by one.
# DOES NOT COMPRESS ARCHIVE FILES! You can comment out "if [[ $file =~ ^.*\.+(7z|zip|bz2|gz)$ ]]; then" and it will.
#
 
# This is called on control-c so we can break from the while loop that is fed with the find command
function breakProgram {
  echo "User issued break signal. Exiting with status 130..."
  exit 130
}
# Catch control-c and call the break function
trap breakProgram SIGINT SIGTERM
 
# USAGE
function usage {
cat << EOF
  usage: $0 [OPTIONS] directory (options must preceed directory)
 
  SCRIPT FOR COMPRESSING FILES AND REMOVING ORIGINALS RECURSIVELY WITH 7ZIP
 
  OPTIONS:
     -h      Show this message
     -p      No prompts
     -s      Silent mode, output only totals of successes and failures
     -d      Don't remove files after compression
 
EOF
}
 
# START ARGUMENT CHECKS
 
# Argument variables
VERBOSE=true
PROMPT=true
DELETE_ON_SUCCESS=true
 
while getopts ?hpsd? OPTION
do
  case $OPTION in
    h)
      usage
      exit 1
      ;;
    p)
      PROMPT=false
      ;;
    s)
      VERBOSE=false
      ;;
    d)
      DELETE_ON_SUCCESS=false
      ;;
    ?)
      usage
      exit
      ;;
  esac
done
 
# This will allow us to get the directory argument
shift $(( OPTIND-1 ))
 
# END ARGUMENT CHECKS
 
# START VARIABLES
 
# Current working directory
WORKING_DIR="`pwd`"
# User supplied directory
TARGET_PARAM="$1"
# Check for absolute path (this is used in the confirmation dialog)
if [[ $TARGET_PARAM == /* ]]; then
  TARGET_DIRECTORY=$TARGET_PARAM
else
  TARGET_DIRECTORY="$WORKING_DIR/"$TARGET_PARAM
fi
 
# Additional safety for deleting files
PROMPT_ON_DELETE=false
 
# Counters for success and failure + failure log
SUCCESS_COUNTER=0
FAILURE_COUNTER=0
FAILURE_LOG=""
 
# 7zip program arguments
PROGRAM_7ZIP="7z a"
#PROGRAM_ARGUMENTS="-xr!*.7z -xr!*.bz2 -xr!*.gz -xr!*.zip"
PROGRAM_7ZIP_TEST="7z t"
if [ $PROMPT == false ]; then
  PROGRAM_ARGUMENTS_EXTRA="-y"
fi
 
# END VARIABLES
 
# START FUNCTIONS
 
# Actual archiving and removal function
function sevenZipIt {
  if [ $VERBOSE == false ]; then
    $PROGRAM_7ZIP "$1.7z" "$1" $PROGRAM_ARGUMENTS_EXTRA 2>&1> /dev/null
  else
    $PROGRAM_7ZIP "$1.7z" "$1" $PROGRAM_ARGUMENTS_EXTRA
  fi
  STATUSCODE=$?
  if [ $STATUSCODE -eq 0 ]; then
    # Retain file attributes
    retainFileAttributes "$1" "$1.7z"
    # Test archive
    $PROGRAM_7ZIP_TEST "$1.7z" $PROGRAM_ARGUMENTS_EXTRA
    STATUSCODE=$?
    if [ $STATUSCODE -eq 0 ]; then
      if [ $DELETE_ON_SUCCESS == true ]; then
        # Remove the original file. Skip prompt so we don't have to answer everytime
        if [ $PROMPT_ON_DELETE == false ]; then
          rm -f "$1"
        else
          rm -i "$1"
        fi
        STATUSCODE=$?
        if [ $VERBOSE == true ]; then
          if [ $STATUSCODE -eq 0 ]; then
            echo "Deleted original file after successful testing of achive: $1"
          else
            echo "Did not delete original file, either user answered no or there was a problem: $1"
          fi
        fi
      fi
      SUCCESS_COUNTER=$(( $SUCCESS_COUNTER + 1 ))
    else
      if [ $VERBOSE == true ]; then
        echo "Testing archive failed, did not delete original file: $1"
      fi
      FAILURE_COUNTER=$(( $FAILURE_COUNTER + 1 ))
      FAILURE_LOG="$FAILURE_LOG\n$1"
    fi
  else
    if [ $VERBOSE == true ]; then
      echo "Archiving failed with status code $statuscode for file: $1"
    fi
    FAILURE_COUNTER=$(( $FAILURE_COUNTER + 1 ))
    FAILURE_LOG="$FAILURE_LOG\n$1"
  fi
}
 
# Retains permissions and ownership. First argument is the original file, second the new file
function retainFileAttributes {
  chmodparam=$( stat --format=%a "$1" )
  chownparam=$( stat --format=%u "$1" )":"$( stat --format=%g "$1" )
  chmod $chmodparam "$2"
  chown $chownparam "$2"
}
 
# END FUNCTIONS
 
# START PROGRAM LOGIC
 
if [ ! -d "$TARGET_DIRECTORY" ]; then
  echo "ERROR Target directory $TARGET_DIRECTORY does not exist! Exiting..."
  exit 1
fi
 
# Ask user for confirmation
if [ $PROMPT == true ]; then
  read -p "Are you sure you want to archive and remove original files from $TARGET_DIRECTORY? " -n 1 -r
  echo
else
  REPLY="y"
fi
 
if [[ $REPLY =~ ^[Yy]$ ]]; then
  # Extra security for delete
  if [ $PROMPT == true ]; then
    read -p "Do you want to be asked before each file removal? " -n 1 -r
    echo
  else
    REPLY="n"
  fi
  if [[ $REPLY =~ ^[Yy]$ ]]; then
    PROMPT_ON_DELETE=true
  fi
 
  # 
  # Go through all the files from the find commands output
  #
  while read file; do
    # Check for undesireable file types
    if [[ $file =~ ^.*\.+(7z|zip|bz2|gz)$ ]]; then
      if [ $VERBOSE == true ]; then
        echo "Skipping archive..."
      fi
    else
      # Call the archiving function
      sevenZipIt "$file"
    fi
  done <<< "`find -P \"$TARGET_DIRECTORY\" -type f -print`"
 
  echo
  echo "Successfully archived and removed $SUCCESS_COUNTER files."
  echo "Failed to archive and remove $FAILURE_COUNTER files."
 
  # Ask to see failure log
  if [ $FAILURE_COUNTER -gt 0 ]; then 
    if [ $VERBOSE == true ]; then
      if [ $PROMPT == true ]; then
        read -p "Do you want to view the log for failed files?" -n 1 -r
        echo
      else
        REPLY="y"
      fi
      if [[ $REPLY =~ ^[Yy]$ ]]; then
        echo
        echo -e "$FAILURE_LOG"
      fi
    fi
  fi
fi
 
# END PROGRAM LOGIC
 
exit 0

There’s a lot more that can be added, but it’s working for me as it is.

Posted in Unix.

Tagged with , , , .


0 Responses

Stay in touch with the conversation, subscribe to the RSS feed for comments on this post.



Some HTML is OK

or, reply to this post via trackback.