KWTm's Journal: detecting subtle changes in Terms & Conditions

This is to check for subtle changes in text that you see often, such as Terms & Conditions that pop up every time you use a frequently-used Web service. It will alert you to small changes that you might not otherwise notice because you habitually click on the "I Agree To These Terms And Conditions" button without going through all the text each time. Simply select the text and copy to the clipboard, and then run the script.

This shell script compares what's in the clipboard to text files in a certain directory; in this case it's ~/Documents/terms_and_conditions. This is where I would store the T&C text from various web sites. When it finds a similar match, it does a diff to look for minor changes. If there is no exact match, it offers to store the copied text in a new text file so you can compare the current version to future versions.

It works with the KDE clipboard, Klipper. Those of you who use GNOME or some other clipboard system will need to modify the line that says: "dcop klipper klipper getClipboardContents >$TEMP_FILE". (In fact, I think those of you using KDE 4 will need to modify this line, too.)

#!/bin/sh
 
COMPARISON_DIR="$HOME/Documents/terms_and_conditions"
COMPARISON_LENGTH=160
SAVE_FOR_FUTURE_FN="clipboard_saved"
TEMP_FILE="temp.tmp"
TEMP_FILEHEAD="temp_head.tmp"
TEMP_CANDIDATE_HEAD="temp_cand_head.tmp"
 
QUIET_FLAG=`echo " $@" | grep -cE " -Q"`
IMMEDIATE_FLAG=`echo " $@" | grep -cE " -I"`
 
if [ $QUIET_FLAG -le 0 ] ; then :
    # The Quiet cmd line argument was NOT specified.
    echo " "
    echo " CompareTermsAndConditions [options] [-I] [-Q] (cannot specify '-IQ' together; '-I -Q' is ok. )"
    echo ": This compares what's in the KDE klipboard with files in $COMPARISON_DIR"
    echo ": automatically searching all files to see which file might be similar or identical."
    echo ": This is to look for changes in those Terms & Conditions on web sites which show them repeatedly, to make sure that after you stopped bothering to check for changes, they don't sneak in unnoticeable changes to the Terms & Conditions."
    echo ": Make sure you have copied the text to clipboard."
    echo ": '-Q' will suppress output to stdout; '-I' will skip the prompt and proceed immediately"
    if [ $IMMEDIATE_FLAG -le 0 ] ; then :
        # The immediate line argument was NOT specified.
        read -p "> ENTER to continue, or Ctrl-C to abort" RESULT
        echo "\n"
    fi
fi
 
OLD_DIR=$PWD
cd $COMPARISON_DIR
 
dcop klipper klipper getClipboardContents >$TEMP_FILE
PATTERN=`grep --max-count=1 '[A-Za-z]' <$TEMP_FILE`
echo ": We will look for files that start with '$PATTERN'"
FILELIST=`grep --fixed-strings -- "$PATTERN" * | sed -r -e 's/([^:]*):.*$/\1/' | uniq`
echo ": Will compare your text to the following similar-looking files:"
echo $FILELIST
 
FOUND_IDENTICAL="False"
for FN in $FILELIST; do :
    if [ $TEMP_FILE != $FN -a $TEMP_FILEHEAD != $FN -a $TEMP_CANDIDATE_HEAD != $FN ]; then :
        #echo " "
        head --bytes=$COMPARISON_LENGTH $TEMP_FILE > $TEMP_FILEHEAD
        head --bytes=$COMPARISON_LENGTH $FN > $TEMP_CANDIDATE_HEAD
        diff --ignore-all-space --brief $TEMP_FILEHEAD $TEMP_CANDIDATE_HEAD > /dev/null
        if [ $? -eq 0 ]; then :
            # Yes, $FN is really worth comparing.
            # We check this just to make sure it's not spuriously matching blank lines in the cliptext.
            echo ": Now comparing with $FN"
            diff --report-identical-files --ignore-all-space $TEMP_FILE $FN
            if [ $? -eq 0 ]; then :
                FOUND_IDENTICAL="True"
                echo ": --- IDENTICAL! ---"
                echo " "
            fi
        else :
            echo ": Skipping $FN, which isn't really all that similar."
        fi
    fi
done
 
if [ "False" = $FOUND_IDENTICAL ]; then :
    echo " "
    if [ $IMMEDIATE_FLAG -le 0 ] ; then :
        # Immediate flag was NOT specified, so we can be interactive and ask if wants to save text to file for future comparison
        echo "! NO MATCH. Want to save the current text for future comparison? (Specify '-I' next time to automatically say yes.)"
        read -p "> ('y'=yes, else no):" RESULT
        if [ "a_y" != a_$RESULT ]; then :
            exit # Otherwise proceed to save
        fi
    fi
    TIMESTAMP=`date +%y%m%d%H%M%S`
    mv $TEMP_FILE "$SAVE_FOR_FUTURE_FN-$TIMESTAMP.txt"
    echo ": Now saving to file $PWD/$SAVE_FOR_FUTURE_FN-$TIMESTAMP.txt"
fi