Become a fan of Slashdot on Facebook

 



Forgot your password?
typodupeerror
User Journal

Journal: GPG signature: first remove whitespace+non-printables 1

Journal by KWTm

I've settled on a hopefully credible way to sign my posts without needing to worry about how word-wrapping will mess up the GPG signature. Before signing my posts, I will remove spaces, tabs, newlines, and any other non-printable characters before signing.

That means that the GPG signature of my posts will be invalid as-is (since the text of the posting itself retains the whitespace and non-printables). That means you will have to remove the whitespace/non-printables yourself before running through the GPG verification to see that it is a valid signature. This is how to do it:

put the plain text into PlainTextFile, and the signature (including beginning and end lines) into SignatureFile, and use this command:
tr -cd [:graph:] <PlainTextFile | gpg --verify SignatureFile -

Don't forget the single hyphen at the end of the line, which has a space before it.

User Journal

Journal: Looking for a linux laptop

Journal by chimpo13

I'd like at least a 15" screen so I can watch movies when I'm in other countries. I'd also like to use Skype so a web cam would be good. Any ideas? It's hard to find laptops with linux installed hard to find. I could roll my own, but I'd rather not have to deal with getting everything to work correctly.

User Journal

Journal: New OpenPGP key as of 2010-01-15

Journal by KWTm

Okay, this time I'm remembering to post my new key before my old key expires. So, this replaces the key that I posted in 2008.

The idea is that, if I get kicked out of this account (or someone takes over this account), then I can set up a new Slashdot account and use this key to prove that I'm the same user.

Here it is. In case Slashdot formats it, remember that if there are any extra-long lines, those should be broken up into individual lines. Each line of this OpenPGP key is the same length and contains no spaces.

-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v1.4.6 (GNU/Linux)

mQGiBEtQZNgRBADGdWKqfED1ZHpS+FbcTF5yq0IpaK6YFrUOxtvY6mhNqBFG/+9M
NyZQxQK9Bfx38KBkeu8H4AHeDHnx19wfrRidS2O/6yrcwahN9i/ZAxF2VgjZafZe
PuYnBXCnIkiPcwzBXOo5MkwKHdqKv6yQYbMWBQolVZLJaxG3W7MAGrHF3wCgpXPu /Vm080e3HWIo46C4GyFe9jkEALPNDenlhnQzTC72nhgEbGdVaYZK5k11gw7kaCSZ
YQougUbBzHZX6yrdfvP7qgtudtpUGNgr4CQc56A+2iTn1eibyw/q/FjlW0IcVXez
f3aIxRMbQO+5vXXnDbED5/dZQvSPNuXzR5q25arRYbaMcoH0YzJSQLg/vkbSHwmn
pKkNA/wOBZl3hb23x56WqPaXb0xyHnY6u3/EPd9Lpn54A0vBcvSqzeXIgrESv6kz
bE7C2FMkp+hLP91brPi51gioSrFs6dDrnRw+adUBMjbApjyCy8XN3wk2IV2Oijud
ca9cFo7Ijsp7oXA4B1S8rvtjc/t1uqiC1VIUtiJJ6cCzQQ6ry7Q2S2EgV2FpIFRh
bSAoMjAxMCBKYW4gMTUgS21haWwpIDxrYXdhaS1rYkB0YW1seWxpbi5uZXQ+iGYE
ExECACYFAktQZNgCGyMFCQWjmoAGCwkIBwMCBBUCCAMEFgIDAQIeAQIXgAAKCRAu
dz05U7/JnhBXAJ4namsGw21iZtLo0TsoYlcnmHRyWwCfZuV4cnJFDus6SjVQ0I1S
4ykXq765Ag0ES1Bk5BAIAPrxJ0ljf/g6cmGtqclwe1UBRHusFapl7wjR4TVoysaU
MB7wD8m41gSSKL4LlirlFkL+EceKr7AVxev2af5qgjV+2Px4TS5VvWcFvd1jPcMs
G8D0j9MNM+lE5SSfOwl/gqcaQoMdTsgx21zsDlgJyithiZbWG0mc9UCufdM8+MeE
BpFiykd9rUcuMSMUaYI5dHSZSv+3ZOqpYC1YI4SRYDck/t13GpcA+hElxiJ3PwWv
0PIfb9QkT3qwICgAJ+E28qjrLHqdGy2bALp/HSlLf2QbwUUj6d6DH7lRXzuV0ZqQ
MI7bnouag0b+bIEWUfK7muyEY1P7Nh5WDLmQ1Ta3mDsAAwYH/j1uwBUjpAmj0SiY
pIwVl+nN0fCyVU45gm7CasBENAJ7jG+ZalK83etZ87jhd/T/pIuKvOso1V1g3zAv
O7BVfXbm94ezE9t2tuv/k9QQoMqfajflBv8rHLzPdyzhc5RaAGbV3fP0+7Q5q01W
lpjs4Dkiau/bdg1B7XBdmb8rovB0gfvD+EhWRr+VWWAAV1NdqDKHyYXcdGfRhkjK
lxUDMc7H2l7H1VUoHXnoL1C+8/KOk9RfuKXdIsGztkJrookykOvZ8GTB91eUL2QH
yEfMetNmAG7NzJM2d0NtZyyULRqujPrUcDsJDhD2u66lDH2FElIlT+Zely5FW+aj
WsA9rMiITwQYEQIADwUCS1Bk5AIbDAUJBaOagAAKCRAudz05U7/JnoR8AKCk1m6r
InrF5Gr0XmiJvfi4hAylNwCdGRRJOtJSEogYds0dwvhrrBUv9NA=
=Kmjs
-----END PGP PUBLIC KEY BLOCK-----

Censorship

Journal: DMCA vs Me 2: Electric Boogalo

Journal by chimpo13

I basically sent pair Networks another WTF message and they responded again with telling me to piss up a rope:

Dear Dave,

I apologize for your confusion and I also appreciate your conviction.
However, pair Networks is a web hosting provider, not a law firm. Our job
is not to interpret the DMCA or the law in general, but to follow it.
Lawyers are very expensive, and the bottom line is that our Service Contract
states that we can terminate all or any part of an account for any reason
without notice. In this case, we are giving you fair warning that to keep
your site online, the best thing for you to do is remove the contested
material. While we appreciate your business, We also have to look out for
our best interest. While I understand that it
causes a moral dilemma for you, the bottom line is that, if we receive a
restraining order, we are not going to fight it; we are going to follow our
lawyer's advice and disable the site as a whole. If you want to proactively
avoid this situation, then you may want to look into moving your site to
another provider. We are not trying to push you away with this statement;
you always have the option of simply removing the content in question.

Sincerely,

Gary H.
pair Networks, Inc.

Censorship

Journal: DMCA, Univ of Minnesota, pair Networks vs Me 3

Journal by chimpo13
My ISP, pair Networks, tells me they'll remove my entire site if they get another DMCA request about my posting 17 out of 600 questions from the MMPI. It's the same issue that slashdot has posted before. My problem is, the same attorney, Carl W. Covert, Jr. from NCS Pearson, Inc., has sent my ISP two DMCAs about the same page, two months apart, and pair hasn't had a customer who sent in a non-compliance response.

With my first DMCA, I went to Chilling Effects and used a boilerplate response. I had a copyright attorney check it before sending it. The DMCA says they have 14 days to sue. I asked pair if it was settled after 14 days went by and only received the "we received your response and will respond". I figured if they ignored that, then the issue was settled. After two months, I got another DMCA.

pair removed my page for 14 days and sent an email saying that if they get another DMCA they will remove my entire site. My site covers a Star Trek band, Sacramento punk rock history, and a blog about riding a crappy old motorcycle round the world. Now I'm in Korea, trying not to fight with an ISP that I was happy with, over 17 questions. I know the easy thing is to edit the single page, and I let most things slide, but issues like this are one my windmills. Any suggestions? Is what pair is doing to me even legal?

User Journal

Journal: processing GnuCash files (XML): a Python script 3

Journal by KWTm

I was asked for the Python code I wrote to process GnuCash files. I figured I may as well post it publicly so that more than one person can benefit from the work I put in. I cleaned it up a bit, but if you have any questions, please contact me. My program here is called GnuCash_Fix.

As mentioned in the comments: When GnuCash imports QFX/OFX/QIF files (say, for a bank account or credit card account), each transaction is between 2 accounts but GnuCash can only identify 1 of the accounts, that of the bank account/credit card account in question. If the QFX/OFX/QIF file is imported without further intervention, the other account is marked as "Unbalanced" and is assigned to this account (for example, "Unbalanced-USD" if in US Dollars) after doing Main Menu > Actions > Check & Repair > All transactions.

This program tries to identify the other account for each transaction (e.g. "groceries" or "salary") depending on the description of the transaction and optionally the range of the amount. It also finds two complementary unbalanced transactions and matches them together. E.g.
# transaction #1 on 2003-06-21, for $123.45 from Account A to ...somewhere unknown!!!; and then
# transaction #2 on 2003-06-21, for $123.45 from somewhere unknown ... to Account B.
Gee, I wonder what could have happened with those two seemingly unrelated transactions???? (duhhh...)

Works for GnuCash v2.2.6, and also for earlier versions that I used (I can't tell which versions now, but at least one of them was a v1.x.x version).

The main tricky thing I found out from working with GnuCash files is that the tags contain the colon character (':') which Python's XML libraries don't like, so I used "sed" to convert them to double underscores ("__")

In addition to depending on some publicly available software such as "sed" and various Python libraries, it also depends on two Python modules of my own creation. I will post these in subsequent entries. It's easy to rewrite the code so it does not have these dependencies, but if you wanted to make sure the code was working as-is, you'll need these modules. One module is just a debugging script (decide whether to display debugging messages depending on a pre-set "debugging level of detail), and the other is just a personalized version of "getopt". (In fact, I think you can just replace it with the official Python "getopt" --but the version with GNU compatibility.)

Oh, it also depends on a pre-defined file containing parameters that allow this GnuCash_Fix program to identify and classify the transactions. This is called descr_acct.py, and I will post it in a subsequent entry.

#!/usr/bin/env python
#
# Copyright KWTm 2006-2009
 
# released under GPL v3. See http://www.gnu.org/copyleft/gpl.html
# Those who don't like this license may contact me for other licensing options, such as my special I-Don't-Like-GPL-And-Want-To-Give-You-Lots-Of-Money license.
 
#
# When GnuCash imports QFX/OFX/QIF files (say, for a bank account or credit card account), each transaction is between 2 accounts but GnuCash can only identify 1 of the accounts, that of the bank account/credit card account in question. If the QFX/OFX/QIF file is imported without further intervention, the other account is marked as "Unbalanced" and is assigned to this account (for example, "Unbalanced-USD" if in US Dollars) after doing Main Menu > Actions > Check & Repair > All transactions.
 
# This program tries to identify the other account for each transaction (e.g. "groceries" or "salary") depending on the description of the transaction and optionally the range of the amount.
#
# Also finds two complementary unbalanced transactions and matches them together. E.g.
# transaction #1 on 2003-06-21, for $123.45 from Account A to ...somewhere unknown!!!; and then
# transaction #2 on 2003-06-21, for $123.45 from somewhere unknown ... to Account B.
# Gee, I wonder what could have happened with those two seemingly unrelated transactions???? (duhhh...)
 
# Works for GnuCash v2.2.6, and also for earlier versions that I used (I can't tell which versions now, but at least one of them was a v1.x.x version).
 
# Dependencies:
# standard Python libraries, especially xml.dom.ext and xml.dom.minidom
# my own modules: kwdebug, kwgetopt (but this script can be easily rewritten to eliminate these dependences)
# standard GNU/Linux command-line utilities: gzip, sed
 
# The Python xml.dom.minidom module doesn't like colons as part of the XML tag name, so we need to s/:/__/ or something.
 
"""
I'll set up a naming convention:
 
_Ac = instance of Account class
_Class = a class
_Cr = instance of Criterion class
_Gn = instance of Gnucash class
_Tr = instance of Transaction class
_Ts = instance of Timestamp class
_Xn = xml_node
 
"""
 
import solitime
import sys
sys.path.append('/usr/lib/python%s/site-packages/oldxml' % sys.version[:3])
# The above 2 lines are necessary because Ubuntu 8.04 moved the python-xml package, breaking old code. Stupid.
import xml.dom.minidom
import xml.dom.ext
 
import kwdebug
MyDebug = kwdebug.Debug(0)
 
class Global_values_Class:
 
    import kwgetopt
    cmd_line_opts = kwgetopt.kwgetopt("a:d:i:o:")
    # Remember: the colon after the option letters mean that there is an argument after those options
    # -a = account containing unbalanced transactions
    # -d = description / account dictionary file
    # -i = input file
    # -o = output file
    #imbal_acct_re = cmd_line_opts.getopt("-a", r"(?i)imbalance.usd") # The other one is "Unspecified" --make sure it doesn't accidentally refer to "Cu Cdn unspecified", by capitalizing "Unspecified")
 
    descr_acct_fname = cmd_line_opts.getopt("-d", "descr_acct.py")
    input_fname = cmd_line_opts.getopt("-i", "gnucash.xml")
    output_fname = cmd_line_opts.getopt("-o", "gnucash-out.xml")
 
    input_unzipped_fname = input_fname + "_unzipped"
    input_unzipped_nocolons_fname = input_unzipped_fname + "_nocolons"
    output_unzipped_fname = output_fname + "_unzipped"
    output_unzipped_nocolons_fname = output_unzipped_fname + "_nocolons"
 
    preamble=''' GnuCashFIX (with transaction description notes, pair matching and RegExp lookup; updated 954JHq)
: This matches corresponding transactions in Gnucash from a designated account.
: It then takes all the transactions in that account
: and moves them to the appropriate accounts based on the description of
: the transaction.
: (-i) GnuCash input: %s
: (-o) GnuCash output: %s
: (-d) Description -> Account matchfile: %s
: (-a) Account to be fixed will be 'Imbalance-USD'
''' % (input_fname, output_fname, descr_acct_fname )
 
    def __init__(self):
        import os.path
        print(self.preamble)
 
        if os.path.abspath(self.input_fname) == os.path.abspath(self.output_fname) :
            print("\n! *Warning*: output will overwrite the input file %s" % self.input_fname)
        else :
            print(": To force output file to overwrite the input file, use the command-line option '-o %s'" % self.input_fname)
 
        try:
            raw_input("> ENTER to continue, or Ctrl-C to abort")
        except KeyboardInterrupt :
            print("\n! Interrupted by keyboard")
            raise SystemExit
 
Global = Global_values_Class()
 
class GnuCashXML_Class:
 
    # I'll make it so that only this class handles XML, and any other classes do not need to access the XML document at all
 
    acct_denominator_tagname = "act__commodity-scu"
    acct_description_tagname = "act__description"
    acct_id_tagname = "act__id"
    acct_name_tagname = "act__name" # Note that it's abbreviated "act", not "acct", in the GnuCash file
    gnc_acct_tagname = "gnc__account"
    #imbal_acct_re = r"(?i)unspecified"
    #imbal_acct_re = r"Uchq" # for testing only
    split_acct_tagname = "split__account"
    #split_id_tagname = "split__id" # This is not used and may be confused with split_acct_tagname
    split_quantity_tagname = "split__quantity"
    split_value_tagname = "split__value"
    transaction_tagname = "gnc__transaction"
    trn_dateposted_tagname = "trn__date-posted"
    trn_description_tagname = "trn__description"
    trn_id_tagname = "trn__id"
    ts_date_tagname = "ts__date"
 
    # To experiment with XML document in Python IDE, use xml.dom.minidom.parse("/home/kwtm1/finances/gnucash-in.conv")
 
    def __init__(self, in_fname = None):
 
        if None != in_fname:
            MyDebug.debug_msg(": Now converting colons")
            conv_fname = Global.input_unzipped_nocolons_fname
            self.colon_to_double_underscore( in_fname, conv_fname )
            MyDebug.debug_msg(": Now importing %s" % conv_fname )
            self.xmldoc = xml.dom.minidom.parse( conv_fname )
        else:
            self.xmldoc = xml.dom.minidom.Document()
 
        self.acct_dict_AcDict = AccountDict_Class({})
 
    def colon_to_double_underscore(self, in_fname, out_fname):
        conv_cmd = "sed -e 's/:/__/g' %s >%s"
        import os
        os.system( conv_cmd % (in_fname, out_fname) )
 
    def double_underscore_to_colon(self, in_fname, out_fname):
        conv_cmd = "sed -e 's/__/:/g' %s >%s"
        import os
        os.system( conv_cmd % (in_fname, out_fname) )
 
    def convert_acct_xml(self, ac):
        # See convert_trans_xml to see how ridiculously nested the XML is.
        ac_name_contentsnode_val = ac.getElementsByTagName(self.acct_name_tagname)[0].childNodes[0].nodeValue
        ac_id_contentsnode_val = ac.getElementsByTagName(self.acct_id_tagname)[0].childNodes[0].nodeValue
        return Account_Class( ac_id_contentsnode_val, ac_name_contentsnode_val, ac)
 
    def convert_trans_xml(self, tr_Tr):
        # Convert from the transactions themselves back to XML
 
        tr_Tr.node.getElementsByTagName(self.trn_dateposted_tagname)[0].getElementsByTagName(self.ts_date_tagname)[0].childNodes[0].nodeValue = tr_Tr.date
 
        tr_Tr.node.getElementsByTagName(self.trn_id_tagname)[0].childNodes[0].nodeValue = tr_Tr.trn_id
 
        try:
            tr_Tr.node.getElementsByTagName(self.trn_description_tagname)[0].childNodes[0].nodeValue = tr_Tr.description
        except IndexError:
            MyDebug.debug_msg( "Bloody transaction %s had no description so we can't modify that." % tr_Tr ,2)
 
        tr_acctids_list = tr_Tr.node.getElementsByTagName(self.split_acct_tagname) # NOT the self.split_id_tagname which is the id of the split itself, not the accounts involved in the transaction
        tr_values_list = tr_Tr.node.getElementsByTagName(self.split_value_tagname)
        tr_quantities_list = tr_Tr.node.getElementsByTagName(self.split_quantity_tagname) # In all GnuCash transactions I've seen, value = quantity, so we can get the value from either one; but when we modify, then we must update both elements
        tr_num_accts_between = len(tr_acctids_list)
 
        if ( tr_Tr.num_accts_between != tr_num_accts_between ):
            MyDebug.debug_msg( "Funny... I thought transaction %s was between exactly %s accounts, but the XML says it's between %s accounts." % (tr_Tr, tr_Tr.num_accts, tr_num_accts_between ),1)
        else:
 
            if ( tr_num_accts_between >= 1):
                tr_acctids_list[0].childNodes[0].nodeValue = tr_Tr.acct1_Ac.acctid
                tr_values_list[0].childNodes[0].nodeValue = self.derive_valuetext(tr_Tr.value)
                tr_quantities_list[0].childNodes[0].nodeValue= self.derive_valuetext(tr_Tr.value)
 
            if ( tr_num_accts_between >= 2):
                tr_acctids_list[1].childNodes[0].nodeValue = tr_Tr.acct2_Ac.acctid
                tr_values_list[1].childNodes[0].nodeValue = self.derive_valuetext( -tr_Tr.value) # The second value is negative to balance the first.
                tr_quantities_list[1].childNodes[0].nodeValue= self.derive_valuetext( -tr_Tr.value) # The second value is negative to balance the first.
 
    def convert_xml_trans(self, tr):
 
        """
        # The XML is nested to a ridiculous degree. To get the date-posted of the transaction, we would have to do:
        tr_dateposted_list = tr.getElementsByTagName(self.trn_dateposted_tagname)
        tr_dateposted = tr_dateposted_list[0]
        tr_dateposted_date_list = tr_dateposted.getElementsByTagName(self.ts_date_tagname)
        tr_dateposted_date = tr_dateposted_date_list[0]
        tr_dateposted_date_contentsnode_list = tr_dateposted_date.childNodes
        tr_dateposted_date_contentsnode = tr_dateposted_date_contentsnode_list[0]
        tr_dateposted_date_contentsnode_val = tr_dateposted_date_contentsnode.nodeValue
        """
 
        tr_dateposted_date_contentsnode_val = tr.getElementsByTagName(self.trn_dateposted_tagname)[0].getElementsByTagName(self.ts_date_tagname)[0].childNodes[0].nodeValue
        #dateposted = self.extract_date(tr_dateposted_date_contentsnode_val)
        #MyDebug.debug_msg( "Extracted date %s" % dateposted,4)
        # No. There is no need to interpret the date ... yet. Just store the date string as is. When we have to compare, only then do we need to interpret the date string stored in a Transaction.
 
        tr_id_contentsnode_val = tr.getElementsByTagName(self.trn_id_tagname)[0].childNodes[0].nodeValue
 
        try:
            tr_description_contentsnode_val = tr.getElementsByTagName(self.trn_description_tagname)[0].childNodes[0].nodeValue
        except:
            tr_description_contentsnode_val = ""
            MyDebug.debug_msg( "Bloody transaction %s has no description!?" % tr_id_contentsnode_val ,2)
 
        tr_acctids_list = tr.getElementsByTagName(self.split_acct_tagname) # NOT the self.split_id_tagname which is the id of the split itself, not the accounts involved in the transaction
        tr_num_accts_between = len(tr_acctids_list)
 
        if ( 2 != tr_num_accts_between ):
            MyDebug.debug_msg( "Funny... transaction %s (%s) is not between exactly 2 accounts, but with %s accounts." % (tr_id_contentsnode_val, tr_description_contentsnode_val, tr_num_accts_between ),4)
 
        if ( tr_num_accts_between >= 1):
            tr_acctid1_contentsnode_val = tr_acctids_list[0].childNodes[0].nodeValue
            acct1_Ac = self.acct_dict_AcDict.get(tr_acctid1_contentsnode_val, None) # Get the account, or return None if no such account
        else:
            acct1_Ac = None
 
        if ( tr_num_accts_between >= 2):
            tr_acctid2_contentsnode_val = tr_acctids_list[1].childNodes[0].nodeValue
            acct2_Ac = self.acct_dict_AcDict.get(tr_acctid2_contentsnode_val, None)
        else:
            acct2_Ac = None
 
        tr_splitvalue_contentsnode_val = tr.getElementsByTagName(self.split_value_tagname)[0].childNodes[0].nodeValue
        value = self.extract_value(tr_splitvalue_contentsnode_val)
 
        # Now to create an instance of Transaction_Class based on the values above
        return Transaction_Class(tr, acct1_Ac, acct2_Ac, tr_dateposted_date_contentsnode_val, tr_description_contentsnode_val, tr_num_accts_between, tr_id_contentsnode_val, value)
 
    def derive_valuetext(self, value, denominator=100):
        return "%d/%d" % (value*denominator, denominator)
 
    def extract_value(self, text):
        # The value text is something like "12345/100" which means "$123.45", but we have to divide numerator by denominator.
 
        try:
            return eval( text.strip() + ".0")
            # If we don't add ".0", then we will be doing integer arithmetic
        except:
            # Okay, just evaluating didn't work, so now we actually have to decode the thing
 
            import re
            value_re = "(\D|^)(?P<numerator>\d+)/(?P<denominator>\d+)(\D|$)"
            match_result = re.search( value_re, text)
            MyDebug.debug_msg("Extracting value from %s" % text,5)
            if match_result == None:
                return None
 
            numerator = float( match_result.groupdict("0").get( "numerator" ) )
            # Get the text that matches the groupname "numerator" in the regular expression (if can't find it, default to "0") and convert it to a float(ing point number) in preparation for calculating the value of this transaction.
            denominator = float( match_result.groupdict("100").get( "denominator" ) )
 
            MyDebug.debug_msg("Value is %s over %s" % (numerator,denominator),5)
            return (numerator/denominator)
 
    def get_acct_dict(self):
        MyDebug.debug_msg("Creating dict of all accounts", 1)
        acct_dict = AccountDict_Class({})
        acct_list = self.xmldoc.getElementsByTagName(self.gnc_acct_tagname)
        for acct in acct_list:
            acct_dict.add_account( self.convert_acct_xml(acct) )
 
        self.acct_dict_AcDict = acct_dict
 
    def get_all_trans_list(self):
        MyDebug.debug_msg("Creating list of all transactions", 1)
        trn_list = self.xmldoc.getElementsByTagName(self.transaction_tagname)
        all_trans_list = Trans_list_Class([])
        for trn in trn_list:
            all_trans_list.append( self.convert_xml_trans(trn) )
        return all_trans_list
 
    def get_unbal_trans_list(self):
        raise NotImplementedError, "Hey, this function is not going to be used."
        #stub
        return []
 
    def modify_original_trans(self, trans_list_TrList):
        for trans_Tr in trans_list_TrList:
            if trans_Tr.need_to_delete_original():
                MyDebug.debug_msg("I would delete %s." % (trans_Tr) ,4)
                trans_Tr.node.parentNode.removeChild(trans_Tr.node)
                trans_Tr.node.unlink()
            elif trans_Tr.need_to_update_original():
                MyDebug.debug_msg("I would update %s." % (trans_Tr) ,4)
                self.convert_trans_xml(trans_Tr)
            else:
                MyDebug.debug_msg("No changes in %s." % (trans_Tr) ,4)
        #stub
 
    def save_changes(self):
        #stub
        pass
 
    def write_to_file(self, out_fname):
        conv_fname = Global.output_unzipped_nocolons_fname
        MyDebug.debug_msg("Writing to file %s" % conv_fname ,1)
        xml.dom.ext.PrettyPrint(self.xmldoc, open(conv_fname, "wb"))
        MyDebug.debug_msg("Converting to %s" % out_fname ,1)
        self.double_underscore_to_colon(conv_fname, out_fname)
 
class Account_Class():
 
    acctid = None
    acctname = None
    acctnode = None
 
    def __init__(self, acctid=None, acctname=None, acctnode=None):
        self.acctid = acctid
        self.acctname = acctname
        self.acctnode = acctnode
 
    def __eq__(self, other):
        if type(other) == type(None):
            return False
 
        if type(other) != type(self):
            raise TypeError, "%s needs to be of type %s" % (other, type(self))
 
        return self.acctid == other.acctid
 
    def __ne__(self, other):
        return not (self == other)
 
    def __repr__(self):
        return "Account %s, which is named %s and resides in node %s" % (self.acctid, self.acctname, self.acctnode)
 
class AccountDict_Class(dict):
    # The dictionary key shall be the account id; the value corresponding to that key shall be an instance of Account_Class that has info about that account.
 
    def __init__(self, *other_args):
        dict.__init__(self, *other_args)
 
    def add_account(self, account):
        self[ account.acctid ] = account
 
    def lookup_by_re(self, name_re):
        # Yes, partial matches do match (ie. if name_re is "bcd" it will find account "abcde")
        import re
        for acct in self.values():
            match_result = re.search( name_re, acct.acctname )
            if match_result != None:
                break
 
        if match_result == None:
            # We didn't find it. We got here because the loop ended, not because we found it and broke out of the loop
            acct=None
 
        return acct
 
class Criteria_Class:
 
    def __init__(self, criteria):
        import re
 
        self.re_c = None
        self.lower_lim = None
        self.upper_lim = None
 
        re_text = "^$" # Initialize to a regular expression that represents the null string
        if str == type(criteria):
            re_text = criteria
        elif tuple == type(criteria) or list == type(criteria):
            if 3 <= len(criteria):
                (re_text, self.lower_lim, self.upper_lim) = tuple(criteria)[:3]
            elif 2 == len(criteria):
                (re_text, self.lower_lim) = tuple(criteria)[:2]
                self.upper_lim = self.lower_lim
            elif 1 == len(criteria):
                re_text = criteria[0]
            # If 0 == len(criteria_text), then we can just use the default values, so we don't have to do anything.
 
        self.re_c = re.compile(re_text)
 
    def __repr__(self):
        return "<criteria: re_c %s, lower_lim %s, upper_lim %s>" % (self.re_c, self.lower_lim, self.upper_lim)
 
    def fulfills_limit_criteria(self, trn_Tr):
 
        # Based on the description of the transaction, such as "MEGA FUEL",
        # as well as upper/lower limits of the transaction amount,
        # figure out whether a given transaction fulfills the criteria
 
        # The order of these tests matter as we go through trying to decide whether the transaction fulfills criteria
 
        #MyDebug.debug_msg("crit.re_c is %s" % self.re_c,3)
        # First check that there's a valid description in the criterion
        if None == self.re_c:
            return False
 
        # Now see if the criterion description matches that of the transaction
 
        if None == self.re_c.search(trn_Tr.description):
            MyDebug.msg_no_cr("N",4)
            return False
        MyDebug.debug_msg("crit matches %s" % trn_Tr.description,4)
        # Now see if there's a valid upper limit (if there's just a lower limit, the upper limit should already have been set to equal the lower limit)
        if None == self.upper_lim:
            MyDebug.msg_no_cr("-",4)
            return True
        elif trn_Tr.value > self.upper_lim:
            MyDebug.msg_no_cr(">",4)
            return False
        elif trn_Tr.value < self.lower_lim:
            MyDebug.msg_no_cr("<",4)
            return False
        else:
            MyDebug.msg_no_cr("=",4)
            return True
 
class Criteria_list_Class(list):
    # contains these properties:
    # crit_dict = the compiled version of the critera. Key = compiled regexp; Value = the account
    # text_dict = the text of the criteria, in a dictionary, read from file
 
    def __init__(self, descr_fname=None, account_dict=None, *other_args):
        list.__init__(self, *other_args)
 
        if descr_fname != None:
            print(": Reading account description from file %s" % descr_fname)
            try:
                self.text_list = self.eval_py_expr_file(descr_fname)
            except TypeError, errmsg:
                print("\n! ERROR: %s" % errmsg)
                print("! Looks like account description file %s is not a correct Python expression. Most likely there is a comma missing at the end of a line." % Global.descr_acct_fname)
                print("! Try looking for the regexp '^[^#]*\)[^,]*($|#)'")
                print("! (that is, a line where a comma does not occur between the closing parenthesis and the end of line/start of comment)")
                raise SystemExit
            except SyntaxError, errmsg:
                print("\n! ERROR: %s" % errmsg)
                print("! Account description file %s is not a correct Python expression. This is not just a comma missing at the end of a line; some extra character got inserted or something." % Global.descr_acct_fname)
                print("! Try deleting parts of the file and retrying, to narrow down where the problem might be.")
                raise SystemExit
 
        else:
            self.text_list = []
 
        if account_dict != None:
            self.acct_dict = account_dict
        else:
            self.acct_dict = AccountDict_Class({})
 
        MyDebug.debug_msg(": initialized crit list" ,2)
        self.initialize_crit_list()
 
    def eval_py_expr_file(self, fname):
        file = open(fname, 'rb')
        py_expr = file.read()
        return eval(py_expr)
 
    def initialize_crit_list(self):
        import re
 
        try:
            for text_list_element in self.text_list:
                (crit_key, acctname_re) = text_list_element[:2]
                acct_Ac = self.acct_dict.lookup_by_re(acctname_re)
                if acct_Ac == None :
                    print("\n! Unidentified account '%s' --ignoring criteria '%s'" % (acctname_re, crit_key))
                else :
                    criteria_Cr = Criteria_Class(crit_key)
                    self.append( (criteria_Cr, acct_Ac) + tuple(text_list_element[2:]) )
                    # Add the criteria and the account name, but also any other elements in the tuple which might correspond to other features
 
        except ValueError, errmsg:
            print("\n! ERROR: %s" % errmsg)
            print("! Is the account description file %s in the wrong format? As of 2009-05-03, it needs to be a list, not a dictionary." % Global.descr_acct_fname)
            raise SystemExit
 
class Timestamp_Class(solitime.Solitime_class):
 
    def __init__(self, *other_args):
        return solitime.Solitime_class.__init__(self, *other_args)
 
    def formatted_timestamp(self, *other_args):
        return solitime.Solitime_class.formatted_solitime(self, *other_args)
 
class Transaction_Class():
 
    node = None
    acct1_Ac = None
    acct2_Ac = None
    #book_Gn = None
    date = None
    description = None
    num_accts_between = 0
    trn_id = None
    value = None
 
    def __init__(self, node=None, acct1_Ac=None, acct2_Ac=None, date=None, description=None, num_accts_between=0, trn_id=None, value=None):
        self.node = node
        self.acct1_Ac = acct1_Ac
        self.acct2_Ac = acct2_Ac
        self.date = date
        self.description = description
        self.num_accts_between = num_accts_between
        self.trn_id = trn_id
        self.value = value
        self.update_flag = False # If this is None, then the transaction is to be deleted
 
    def __repr__(self):
        return "Transaction, with id %s, for amt$ %s, on date %s, between acct %s and acct %s, covering %s accounts, described as %s. Stored in XML node %s." % (self.trn_id, self.value, self.date, self.acct1_Ac, self.acct2_Ac, self.num_accts_between, self.description, self.node)
 
    def __str__(self):
        return "Transaction %s: on %s, $%s from %s to %s." % (self.trn_id, self.date, self.value, self.acct1_Ac.acctname, self.acct2_Ac.acctname)
 
    def extract_timestamp(self):
        mytimestamp_Ts = Timestamp_Class()
        mytimestamp_Ts.set_time_from_string(self.date)
        MyDebug.debug_msg("Converting date from '%s' to '%s'" % (self.date, mytimestamp_Ts.formatted_timestamp()) ,4)
        return mytimestamp_Ts
 
    def flag_to_delete_original(self):
        self.update_flag = None
 
    def flag_to_preserve_original(self):
        self.update_flag = False
 
    def flag_to_update_original(self):
        self.update_flag = True
 
    def need_to_delete_original(self):
        return ( None == self.update_flag)
 
    def need_to_update_original(self):
        return ( True == self.update_flag)
 
class Trans_list_Class(list):
 
    def __init__(self, *other_args):
        list.__init__(self, *other_args)
 
    def find_matching_trans(self, unbal_trans_Tr):
        #found_it = False # We don't need this yet
        for poss_matching_trans_Tr in self:
            if self.the_dates_match(unbal_trans_Tr, poss_matching_trans_Tr) and \
                self.the_accounts_and_amounts_match(unbal_trans_Tr, poss_matching_trans_Tr):
                return poss_matching_trans_Tr
 
        return None
 
    def the_accounts_and_amounts_match(self, unbal_trans_Tr, poss_matching_trans_Tr):
        # Two cases: both transactions share the same Acct1, or same Acct2; then first amount should be negative of the second amount
        # Otherwise if the Acct1 of one is the same as the Acct2 of the other, or vice versa, then the amounts should be identical (not negatives of each other)
        if (unbal_trans_Tr.acct1_Ac == poss_matching_trans_Tr.acct1_Ac) or (unbal_trans_Tr.acct2_Ac == poss_matching_trans_Tr.acct2_Ac):
            return unbal_trans_Tr.value == -(poss_matching_trans_Tr.value)
        elif (unbal_trans_Tr.acct1_Ac == poss_matching_trans_Tr.acct2_Ac) or (unbal_trans_Tr.acct2_Ac == poss_matching_trans_Tr.acct1_Ac):
            return unbal_trans_Tr.value == poss_matching_trans_Tr.value
 
    def the_dates_match(self, unbal_trans_Tr, poss_matching_trans_Tr):
        unbal_trans_timestamp = unbal_trans_Tr.extract_timestamp()
        poss_matching_trans_timestamp = poss_matching_trans_Tr.extract_timestamp()
 
        return_val= abs(unbal_trans_timestamp.time_tuple[0]-poss_matching_trans_timestamp.time_tuple[0]) <= 1 and \
            unbal_trans_timestamp.time_tuple[1] == poss_matching_trans_timestamp.time_tuple[1] and \
            unbal_trans_timestamp.time_tuple[2] == poss_matching_trans_timestamp.time_tuple[2]
        # The dates will be considered matching if the month and day match exactly, and if the years match or are exactly 1 apart (my own special code)
        return return_val
 
class Transfix_Class():
 
    def __init__(self):
        pass
 
    def classify_trans_by_descr(self, trn_Tr):
        # Based on the description of the transaction, such as "MEGA FUEL",
        # figure out which income/expense account this should be, such as "Expenses:consumables:gasoline"
        #description_node = trn.getElementsByTagName(Global.trn_description_tagname)
        #if 0 >= len(description_node):
        # raise AccountError, "Can't find transaction description"
        #description = description_node[0].childNodes[0].nodeValue
 
        return_acct_Ac = None
        return_trans_descr = None
        for element in self.criteria_list_CrList:
            (criterion_Cr, acct_Ac) = element[:2]
            if criterion_Cr.fulfills_limit_criteria( trn_Tr ):
                # The transaction description matches, so let's see what account this transaction should be under
                return_acct_Ac = acct_Ac
                if len(element) >=3:
                    return_trans_descr = element[2] + " - " + trn_Tr.description
                    # ToDo: change this to a more sophisticated substitution
                break
 
        return (return_acct_Ac, return_trans_descr)
 
    def classify_trans_of_translist_by_descr(self, remaining_unbal_trans_list_TrList):
 
        print("")
        for unbal_trans_Tr in remaining_unbal_trans_list_TrList:
            (corrected_acct_Ac, corrected_trans_descr) = self.classify_trans_by_descr(unbal_trans_Tr)
            if None != corrected_acct_Ac:
                # Do this only if we've actually found the corrected acct id, or else leave the acct id alone
 
                # But is it acct1 or acct2 of the transaction that we need to correct? The following takes care of this.
                if unbal_trans_Tr.acct1_Ac in self.unbal_acct_list:
                    unbal_trans_Tr.acct1_Ac = corrected_acct_Ac
                elif unbal_trans_Tr.acct2_Ac in self.unbal_acct_list:
                    unbal_trans_Tr.acct2_Ac = corrected_acct_Ac
 
                if corrected_trans_descr != None:
                    unbal_trans_Tr.description = corrected_trans_descr
 
                unbal_trans_Tr.flag_to_update_original()
                MyDebug.msg_no_cr("!", 8)
                print(": (%s) $%0.2f\t%s -> '%s'" % (unbal_trans_Tr.date, unbal_trans_Tr.value, unbal_trans_Tr.description, corrected_acct_Ac.acctname))
            else:
                print("! (%s) $%0.2f *\t*\t* no account found for '%s'" % (unbal_trans_Tr.date, unbal_trans_Tr.value, unbal_trans_Tr.description))
 
    def extract_unbal_trans_list(self, all_transactions, unbal_acct_list):
        # First, clean up unbal_acct_list because it might contain None's
        no_more_Nones = False
        while not no_more_Nones:
            try:
                unbal_acct_list.remove(None)
            except:
                # We couldn't remove any more Nones from the list, so we must have gotten rid of all of them
                no_more_Nones = True
 
        MyDebug.debug_msg("After removing Nones, our unbal_acct_list is %s" % unbal_acct_list,4)
 
        unbal_trans_list = Trans_list_Class([])
        for tr in all_transactions:
            MyDebug.msg_no_cr("t", 3)
            #MyDebug.debug_msg("Checking tr %s." % tr,4)
            for unbal_acct in unbal_acct_list:
                MyDebug.debug_msg("Checking with account %s." % unbal_acct,4)
                if (tr.acct1_Ac == unbal_acct) or (tr.acct2_Ac == unbal_acct):
                    MyDebug.debug_msg("Found that transaction %s is unbalanced." % tr, 3)
                    unbal_trans_list.append(tr)
 
        return unbal_trans_list
 
    def find_matching_pairs_of_unbal_trans(self, unbal_trans_list_TrList):
    #def get_book_fn(self):
    # return "gnucash.xml"
        self.matched_unbal_trans_list = [] # This will contain tuples (pairs) of the matched accounts
        self.unmatched_unbal_trans_list_TrList = Trans_list_Class([])
        self.already_matched_unbal_trans_list_TrList = Trans_list_Class([])
        for unbal_trans_Tr in unbal_trans_list_TrList:
            if unbal_trans_Tr not in self.already_matched_unbal_trans_list_TrList :
                match_unbal_trans_Tr = unbal_trans_list_TrList.find_matching_trans(unbal_trans_Tr)
                if None != match_unbal_trans_Tr:
                    # We found a match! Now put the match in already_matched_unbal_trans_list_TrList so we don't try to look for a match (which would produce duplicate match)
                    self.already_matched_unbal_trans_list_TrList.append( match_unbal_trans_Tr )
                    self.matched_unbal_trans_list.append( (unbal_trans_Tr, match_unbal_trans_Tr) )
                    #my_gnucash_book_Gn.reconcile(unbal_trans_Tr, match_unbal_trans_Tr)
                    MyDebug.debug_msg("Found match. Currently %s unbalanced and %s pairs of matched transactions." % ( len(self.unmatched_unbal_trans_list_TrList),len( self.matched_unbal_trans_list ) ) ,4)
                else:
                    self.unmatched_unbal_trans_list_TrList.append( unbal_trans_Tr )
                    MyDebug.debug_msg("Could not find match for %s" % unbal_trans_Tr,4)
                    MyDebug.debug_msg("No match. Currently %s unbalanced and %s pairs of matched transactions." % ( len(self.unmatched_unbal_trans_list_TrList),len( self.matched_unbal_trans_list ) ) ,4)
        print(": Could not find match for %s unbalanced transactions." % len(self.unmatched_unbal_trans_list_TrList))
        print(": %s pairs of transactions to be reconciled." % len( self.matched_unbal_trans_list ))
 
        for (tr1_Tr, tr2_Tr) in self.matched_unbal_trans_list:
            self.reconcile_matching_unbal_trans(tr1_Tr, tr2_Tr)
 
        return self.unmatched_unbal_trans_list_TrList
 
    def reconcile_matching_unbal_trans(self,tr1_Tr, tr2_Tr):
        # The goal is to modify the transaction where the *second* account (not the first) is the shared one.
        # Then we delete the transaction where the first account is the shared one. So, if we have:
        # tr1: $10 from Acct A to Acct B
        # tr2: $10 from Acct B to Acct C
        # Then we delete tr2, and change tr1 to: $10 from Acct A to Acct C
 
        if (tr1_Tr.acct2_Ac == tr2_Tr.acct1_Ac):
            tr1_Tr.acct2_Ac = tr2_Tr.acct2_Ac
            tr2_Tr.acct1_Ac = tr1_Tr.acct1_Ac
        elif (tr1_Tr.acct2_Ac == tr2_Tr.acct2_Ac):
            tr1_Tr.acct2_Ac = tr2_Tr.acct1_Ac
            tr2_Tr.acct2_Ac = tr1_Tr.acct1_Ac
        elif (tr1_Tr.acct1_Ac == tr2_Tr.acct2_Ac):
            tr1_Tr.acct1_Ac = tr2_Tr.acct1_Ac
            tr2_Tr.acct2_Ac = tr1_Tr.acct2_Ac
        elif (tr1_Tr.acct2_Ac == tr2_Tr.acct2_Ac):
            tr1_Tr.acct2_Ac = tr2_Tr.acct1_Ac
            tr2_Tr.acct2_Ac = tr1_Tr.acct1_Ac
        else:
            raise AssertionError, "Looks like these aren't really matching transactions: %s and %s" % (tr1_Tr, tr2_Tr)
 
        # At this point each of the two transactions where one of the accounts was the Imbalance account has had its Imbalance account replaced with the correct account. So there are 2 duplicate transactions and we need to get rid of one.
        # Which do we keep? The one with the earlier date, since my special code is that I will put a date one year ahead. That is, if I deposit a cheque for $123.45 on 2003-06-21, then I'll put a reminder transaction that on 2004-06-21, $123.45 went somewhere. So even if the cheque doesn't show up right away, I will have a reminder that there is a transaction there... somewhere. I think. Does that work? Whatever. Just do it.
        if tr1_Tr.date < tr2_Tr.date :
            tr1_Tr.flag_to_update_original()
            tr2_Tr.flag_to_delete_original()
        else:
            tr2_Tr.flag_to_update_original()
            tr1_Tr.flag_to_delete_original()
 
        MyDebug.debug_msg("Reconciled transactions to: \n%s, and \n%s" % (tr1_Tr, tr2_Tr) ,4)
 
    def xml_manip_main(self):
        import os
 
        print(": Reading from finances file '%s'" % Global.input_fname)
        MyDebug.debug_msg("Now unzipping")
        result = os.system( "gunzip --to-stdout %s >%s" % (Global.input_fname, Global.input_unzipped_fname) )
        MyDebug.debug_msg("unzip result is %s" % result)
        if result != 0 :
            print(": %s does not appear to be zipped. Trying as unzipped file." % Global.input_fname)
            input_unzipped_fname = Global.input_fname
        else :
            input_unzipped_fname = Global.input_unzipped_fname
 
        self.my_gnucash_book_Gn = GnuCashXML_Class(input_unzipped_fname)
        self.my_gnucash_book_Gn.get_acct_dict()
        self.acct_dict_AcDict = self.my_gnucash_book_Gn.acct_dict_AcDict
        #MyDebug.debug_msg("Formed account dictionary of size %s" % len(acct_dict_AcDict) ,1)
        print(": Found %s accounts in %s" % (len(self.acct_dict_AcDict), Global.input_fname) )
        MyDebug.debug_msg(self.acct_dict_AcDict,4)
 
        self.all_trans_list_TrList = self.my_gnucash_book_Gn.get_all_trans_list()
        #MyDebug.debug_msg("There are %s transactions in total." % len(all_trans_list_TrList) ,1)
        print(": Found %s transactions in %s" % (len(self.all_trans_list_TrList) , Global.input_fname) )
 
        self.unbal_acct_list = [self.acct_dict_AcDict.lookup_by_re("Unspecified"),
                    self.acct_dict_AcDict.lookup_by_re("Imbalance-USD")]
        MyDebug.debug_msg("Unbal acct list is %s" % self.unbal_acct_list,2)
 
        self.unbal_trans_list_TrList = self.extract_unbal_trans_list( self.all_trans_list_TrList, self.unbal_acct_list )
        print(": There are %s unbalanced transactions." % len(self.unbal_trans_list_TrList))
 
        # We will store the original unbal_trans_list,
        # and then do various things like match up corresponding transactions and switch the account to the right account based on the description
        # Each time we do this, we maintain a list of transactions that are still not fixed, and pass it to the next function
 
        # Step 1
        #remaining_unmatched_unbal_trans_list_TrList = self.unbal_trans_list_TrList
        remaining_unmatched_unbal_trans_list_TrList = self.find_matching_pairs_of_unbal_trans(self.unbal_trans_list_TrList)
        #MyDebug.debug_msg(": skipping matching pairs. Revert this later on.")
 
        #remaining_unmatched_unbal_trans_list_TrList = self.fix_account_if_needed_based_on_descr(remaining_unbal_trans_list_TrList)
 
        # Step 2
        self.criteria_list_CrList = Criteria_list_Class(Global.descr_acct_fname, self.acct_dict_AcDict)
        remaining_unmatched_unbal_trans_list_TrList = self.classify_trans_of_translist_by_descr(remaining_unmatched_unbal_trans_list_TrList )
 
        self.my_gnucash_book_Gn.modify_original_trans( self.unbal_trans_list_TrList )
 
        print(": Done. Now writing to file '%s'" % Global.output_fname)
        self.my_gnucash_book_Gn.write_to_file(Global.output_unzipped_fname)
        MyDebug.debug_msg("Now zipping")
        result = os.system( "gzip --to-stdout %s >%s" % (Global.output_unzipped_fname, Global.output_fname) )
        MyDebug.debug_msg("zip result is %s" % result)
        if result != 0 :
            print(": %s may not have been zipped properly. Try using unzipped intermediate file %s" % (Global.output_fname, Global.output_unzipped_fname) )
 
def main():
    mytransfix = Transfix_Class()
    mytransfix.xml_manip_main()
 
if __name__ == "__main__":
    main()

End of listing for my Python program.

User Journal

Journal: detecting subtle changes in Terms & Conditions

Journal by KWTm

This is to check for subtle changes in text that you see often, such as Terms & Conditions that pop up every time you use a frequently-used Web service. It will alert you to small changes that you might not otherwise notice because you habitually click on the "I Agree To These Terms And Conditions" button without going through all the text each time. Simply select the text and copy to the clipboard, and then run the script.

This shell script compares what's in the clipboard to text files in a certain directory; in this case it's ~/Documents/terms_and_conditions. This is where I would store the T&C text from various web sites. When it finds a similar match, it does a diff to look for minor changes. If there is no exact match, it offers to store the copied text in a new text file so you can compare the current version to future versions.

It works with the KDE clipboard, Klipper. Those of you who use GNOME or some other clipboard system will need to modify the line that says: "dcop klipper klipper getClipboardContents >$TEMP_FILE". (In fact, I think those of you using KDE 4 will need to modify this line, too.)

#!/bin/sh
 
COMPARISON_DIR="$HOME/Documents/terms_and_conditions"
COMPARISON_LENGTH=160
SAVE_FOR_FUTURE_FN="clipboard_saved"
TEMP_FILE="temp.tmp"
TEMP_FILEHEAD="temp_head.tmp"
TEMP_CANDIDATE_HEAD="temp_cand_head.tmp"
 
QUIET_FLAG=`echo " $@" | grep -cE " -Q"`
IMMEDIATE_FLAG=`echo " $@" | grep -cE " -I"`
 
if [ $QUIET_FLAG -le 0 ] ; then :
    # The Quiet cmd line argument was NOT specified.
    echo " "
    echo " CompareTermsAndConditions [options] [-I] [-Q] (cannot specify '-IQ' together; '-I -Q' is ok. )"
    echo ": This compares what's in the KDE klipboard with files in $COMPARISON_DIR"
    echo ": automatically searching all files to see which file might be similar or identical."
    echo ": This is to look for changes in those Terms & Conditions on web sites which show them repeatedly, to make sure that after you stopped bothering to check for changes, they don't sneak in unnoticeable changes to the Terms & Conditions."
    echo ": Make sure you have copied the text to clipboard."
    echo ": '-Q' will suppress output to stdout; '-I' will skip the prompt and proceed immediately"
    if [ $IMMEDIATE_FLAG -le 0 ] ; then :
        # The immediate line argument was NOT specified.
        read -p "> ENTER to continue, or Ctrl-C to abort" RESULT
        echo "\n"
    fi
fi
 
OLD_DIR=$PWD
cd $COMPARISON_DIR
 
dcop klipper klipper getClipboardContents >$TEMP_FILE
PATTERN=`grep --max-count=1 '[A-Za-z]' <$TEMP_FILE`
echo ": We will look for files that start with '$PATTERN'"
FILELIST=`grep --fixed-strings -- "$PATTERN" * | sed -r -e 's/([^:]*):.*$/\1/' | uniq`
echo ": Will compare your text to the following similar-looking files:"
echo $FILELIST
 
FOUND_IDENTICAL="False"
for FN in $FILELIST; do :
    if [ $TEMP_FILE != $FN -a $TEMP_FILEHEAD != $FN -a $TEMP_CANDIDATE_HEAD != $FN ]; then :
        #echo " "
        head --bytes=$COMPARISON_LENGTH $TEMP_FILE > $TEMP_FILEHEAD
        head --bytes=$COMPARISON_LENGTH $FN > $TEMP_CANDIDATE_HEAD
        diff --ignore-all-space --brief $TEMP_FILEHEAD $TEMP_CANDIDATE_HEAD > /dev/null
        if [ $? -eq 0 ]; then :
            # Yes, $FN is really worth comparing.
            # We check this just to make sure it's not spuriously matching blank lines in the cliptext.
            echo ": Now comparing with $FN"
            diff --report-identical-files --ignore-all-space $TEMP_FILE $FN
            if [ $? -eq 0 ]; then :
                FOUND_IDENTICAL="True"
                echo ": --- IDENTICAL! ---"
                echo " "
            fi
        else :
            echo ": Skipping $FN, which isn't really all that similar."
        fi
    fi
done
 
if [ "False" = $FOUND_IDENTICAL ]; then :
    echo " "
    if [ $IMMEDIATE_FLAG -le 0 ] ; then :
        # Immediate flag was NOT specified, so we can be interactive and ask if wants to save text to file for future comparison
        echo "! NO MATCH. Want to save the current text for future comparison? (Specify '-I' next time to automatically say yes.)"
        read -p "> ('y'=yes, else no):" RESULT
        if [ "a_y" != a_$RESULT ]; then :
            exit # Otherwise proceed to save
        fi
    fi
    TIMESTAMP=`date +%y%m%d%H%M%S`
    mv $TEMP_FILE "$SAVE_FOR_FUTURE_FN-$TIMESTAMP.txt"
    echo ": Now saving to file $PWD/$SAVE_FOR_FUTURE_FN-$TIMESTAMP.txt"
fi

User Journal

Journal: new OpenPGP key as of 2008-08-01

Journal by KWTm

Well, gee, the plan was to sign this new key before the old key lapsed, but lazy me... anyway, at least I'm still using the same user account. The idea is that, if I get kicked out of this account (or someone takes over this account), then I can set up a new Slashdot account and use this key to prove that I'm the same user.

Here it is. In case Slashdot formats it, remember that if there are any extra-long lines, those should be broken up into individual lines. Each line of this OpenPGP key is the same length and contains no spaces.

-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v1.4.6 (GNU/Linux)

mQGiBEgo38YRBADIxPJE2WJjqCnqp7dshy772sOfpdud15I+pJSRVq8/ED8R6/A/
NFe8GCqL5//IRvj2WaJtd8OLVfJtzb+4Q8kTvgtsBCQU02KyBGEBGhyfxwUBxEYP
Ynwu2GdhkodLRKOEdmLuegvirmxbjFr2Bb408lRG7UkUbWH2uZRTvIoRYwCg8Pib
56ZSgLIRZbip/MLnavMtrKUD/RHCrUDJg09zmN9P0DwaFdod1Tp5xNg/ko3uGli9
0+WjPEMCQCK51nwNEnPijhbRq38f/Jbut/fFJlawwEyrEaufIfGr+FwLX2zKhgSM
a57q/b4dzz9f7Ky4vwoJK+Oh8zlISL+uZs5RPRep+GjYUJTjMa8D+R9txb7gX/tX
17coBAC7Akp8SFiXvbylN1eSzvWUYiwoD7R9m4naLhHpIcf1A6J1H1Tra/ZvsMcp
k+Cz6gy3FrZn2Az6Gi2+gvJE04ma0loKN5kN90tNm9BEM24/25Ontef1ubL47Dty
iY8MNfTFzcTCaapmwGJQ4h017p5Z8hyXFglz4Stdpa7CgK1GsLQ2S2EgV2FpIFRh
bSAoMjAwOCBNYXkgMTIgS21haWwpIDxrYXdhaS1rYkB0YW1seWxpbi5uZXQ+iGYE
ExECACYFAkgo38YCGyMFCQPCZwAGCwkIBwMCBBUCCAMEFgIDAQIeAQIXgAAKCRBx /pIpG5px4O6DAJwNbNQEUakeF6mPRb3Q4T18ZAP1YgCg5LSpeC/tO3SI8g9X7n4F
4K2VGGS5AQ0ESCjfxxAEAKDFoVXqAAOI1LMetFiSqbBxc4cf813955ltLY88qJi8
1bJTWFQe1YFl4Kq/i71gkri9xlkNnERemipsyoHv/i2Pb1FeQvzLm6BnvZqRCqpQ
eRg4TOcaoQ/kFecBTBg4rGwVXG25Jjffr336LJ0/cO0LQGcZxRadAwv3l50XYubX
AAMHBACWK80MNxV9bVjO9RwY7DHfchETq7NtLVFwxjkdPacROfqQSEJ0i1c1eJ+Y
wPt22i9B3tT0g26wtqexKp0GAE6FP0IHrrw8QeI1BEkepvF3IPfVspJNJ89/0e3u
Wvkk7n6QtRbfYKp2M+V6WSh76TgtUdCxG7xNxsQXQYu5e+cK24hPBBgRAgAPBQJI
KN/HAhsMBQkDwmcAAAoJEHH+kikbmnHg/x0AoKRNFGP+WKUZxehMIzutH6lPWr78
AJ9K+Ng5s0mUCvPX1PmnDJG6uqkI4A==
=oRdN
-----END PGP PUBLIC KEY BLOCK-----

User Journal

Journal: How I Learned Philosophy

Journal by chimpo13

From the "Paying People to Argue With You" thread.

http://slashdot.org/article.pl?sid=07/11/05/1353215

How I Learned Philosophy (Score:5, Insightful)
by severoon (536737)
http://slashdot.org/comments.pl?sid=350509&cid=21247745

Actually, it's not the attempt to mathify that I find problematic--I find that encouraging. It is, though, the results.

My (awesome) university philosophy professor had us do a very interesting exercise that was, though more logical than mathematical in nature, similar to what the author of TFA was going for. It goes like this...

Write down a belief that you have. For people new to this process (the entire class), this should be a strongly held belief...doesn't matter how controversial. Let's say, for example: I think abortion should be a woman's choice. (For you controversy-hounds out there, please don't mistake this for my actual belief--I'm intentionally not going to define my actual belief on this topic here.) Don't worry about getting the wording just right--you're free to revisit your initial statement as many times as you like throughout and revise it to more concisely represent your intent.

Now write down the set of "sub-beliefs" that you have which form the basis of your belief. For our example: 1. Life begins at conception. 2. Every life is equally valuable. 3. A life has no quantifiable value, but is inherently precious and ought to be protected if at all possible. Etc. Next we iterate, applying the same process to each belief listed. Obviously, you will very quickly diverge into an explosion of statements that resist corralling at every effort. Do not fret--I haven't told you about the thrust of the exercise yet.

(I should mention here that we did an entire section on identifying context-free statements, and we were asked to make our best effort to ensure that each statement was context-free, or as free of context as possible. "Context-free" means that the statement is true of our beliefs regardless of the circumstances in which the statement is tested. If that's not possible--and it's not often possible--we'd go for "generally" true, where "common sense"--whatever that is--dictates obvious exceptions.)

You will find it unnecessary to list each and every belief supporting your initial statement, which would quite likely fill several thick volumes if you did so exhaustively. Luckily, you don't have to do this to satisfy the point of the exercise, which is: where necessary, skip down to "lowest level" beliefs...that is, at some point you will mentally reach a point where you have identified a belief for which you have no further basis beliefs. When you reach this point, you have identified an axiomatic belief--that is, something you accept essentially on faith, on gut feeling, because you think it is correct. If possible, identify the key beliefs that go from your initial statement to the set of axiomatic beliefs identified.

The next step is to look at your beliefs, both axiomatic and intermediate, for consistency. In every case in carrying out this exercise, one will invariably find a whole host of contradictory statements. Then we did an iteration that attempts to resolve these conflicts by tweaking our initial statement, etc...provided we were tuning up the language to indicate real intent and not moving the statements further away from our actual beliefs, great. The ultimate idea is to identify our beliefs in all their gory, inconsistent, warty detail.

Then, we make up a list of so-called axiomatic beliefs and they are given to 5 random classmates (all double-blind, of course). You then are tasked with taking home those 5 lists of axiomatic beliefs and attempt to drill down further. If they are truly axiomatic, you won't be able to do this--the idea here is that you ultimately get back 5 people's analysis of your list and given another chance to continue the process--most of the time, it turns out you realize your axiomatic beliefs weren't axiomatic for you after all, and that you can actually drill down even more.

Anyway, it goes on like this, the ultimate point being that you arrive at some network of beliefs which you apparently do accept as axiomatic. The focus here is not on the logic that leads you down the path from the initial statement to the final list...the point is to show that your beliefs are not rigorously logical, even after you've done your level best to identify all the logical flaws, ultimately you wind up with a list of axiomatic beliefs that either directly or indirectly contradict each other to some extent. What these beliefs are, where the conflicts are, and how you resolve these conflicts all roughly correspond to your worldview.

As an entertaining add-on at the end of the course, the prof provided us with some very mild statistical metrics that told us how self-contradictory our beliefs were when pegged against our classmates, previous years, different types of statements (it was generally true that the more strongly held / the more controversial the statement, such as the abortion example above, the less self-consistent the foundational beliefs identified).

For a couple of years after doing this exercise, I found it very difficult to make strong statements of opinion about controversial topics. My mind would involuntarily start this process, identifying all the biggest logical hurdles and inconsistencies built into the statements I was about to make. This reflex also made me annoying to others with strongly held beliefs. :-)

User Journal

Journal: ah, the joy of never using emoticons

Journal by chimpo13
You'd figure people would read a comment and react to what you wrote. But no, it's far easier to skim the comment, read it as the complete opposite of what you wrote and then go with it.

I make a comment about how nukes should be sold to everyone. It's quit obvious that's meant to be taken very seriously.

Doc Ruby takes it as I'm so super-duper pro-gun that he starts rambling nonsense in a crazy rant.

I'm not sure exactly what he meant to be saying, but I do enjoy him calling me a gun fetishist. He was modded up to 5 insightful for something that he took completely wrong. Reading over Doc's journal, I noticed that he has a special friend who says when they have mod points, they look for his posts to mark him down. I wonder if he skims and reacts to people often. It reminds me of Bill O'Reilly only he's doing it from the Left Wing point of view. I'm quite a leftist myself but the O'Reillys of the world, right wing or left wing, are all kooks.

It's great how I've been modded to oblivion as a troll, but someone who agrees with what I wrote, only not saying it so sarcastically has been modded up.

User Journal

Journal: My Dellbuntu laptop: installing Kubuntu

Journal by KWTm

Installing Kubuntu:
- the plan is to shrink the built-in Ubuntu partition on /dev/sda6 down to 6GB (my original plan was 4GB, but there's a frigg'n 4.3GB of files in there already! What did they load on there, some voice-recognition stuff?)
- expand the swap partition to 4GB (since there's 2GB of RAM in there)
- then add another 4GB partition to install Kubuntu
- then the remaining 130+ GB is for data

- running from the Kubuntu 7.04 Live DVD, QTParted is unable to resize the main 151GB partition! --can't move, can't resize; I can only choose to delete it if anything. The partition was not mounted, so not sure why I couldn't manipulate it.
- so we need some other tools for managing it, instead of QTParted. I bought PartitionExpert from Acronis (in 2003, it was $45, downloaded from web), which is able to handle Ext3 as well as ReiserFS partitions. At the time, Partition Magic (which was v6 at the time) was not able to do ReiserFS. Anyway, so in this step I was not able to move ahead with any FLOSS disk partition manager known to me. (Any suggestions for other partition managers I should try, in the future?)
- using Partition Expert: surprise! The 4.3GB of data turned out to be only about 1.7GB, taking up less room than I thought. I was able to shrink the gUbuntu partition (/dev/sda6) down to 4GB, create a new 4GB partition for Kubuntu (/dev/sda7), and make the remaining 135GB partition (/dev/sda8) for data.

(Here are partial results for "df -h", listing device, total size, used size, remaining size, %used, and mount point.) /dev/sda1 47M 876K 47M 2% /media/sda1 /dev/sda2 2.0G 693M 1.4G 34% /media/sda2 /dev/sda3 193M 21M 163M 12% /media/sda3 /dev/sda6 4.0G 1.7G 2.1G 45% /media/sda6 /dev/sda7 4.0G 2.8G 1000M 74% / /dev/sda8 135G 3.4G 132G 3% /media/sda8
(/dev/sda4 is not listed. That's the swap partition, which I increased to 4GB.)

- it took a long, LOOOooooo..ng time to do it, though. Using Partition Expert, I first clicked on the "big" partition (at the time it was /dev/sda6, 150GB), just to say, "I'll work with that partition". A window popped up: "analyzing partition", and then it froze. At least, it looked like it. Actually, it really was analyzing the partition, except it took 20 minutes! So, beware not to click on the wrong partition by accident.
- and then once I specified how I wanted the partitions set up, it took another 40 minutes or so. Not sure --it took so long that I lost track.

- Once everything was resized, Kubuntu installed smoothly.
- Previously, installing Kubuntu on my desktop boxes, I did very little manually; instead, I typed commands into a script and then ran the script, eventually accumulating a long record of my installation steps in a shell script. This paid off now as I copied and pasted large chunks of this script into a similar installation script for the Dellbuntu, and it took only about half a dozen steps (with big sections of nothing but "sudo apt-get --assume-yes install" lines) to reproduce my desktop configuration on the Dellbuntu.
- by installing the 915resolution package ("sudo apt-get install 915resolution"), I got to use the full resolution of 1280x800 \
- I previously worried about the screen being too small, but I guess with the full resolution the screen looks pretty big. I'm rather pleasantly surprised. My desktop has a 19" monitor capable of 1280x1024, but the Linux driver for the onboard graphics card can only drive it at 1024x768. I never realized till now how much screen size is dependent on resolution rather than just physical size.
- By the way, I chose the normal matte screen rather than the default Dell TrueLife(tm) Screen With Sharper Colours And Annoying Reflection. I'm glad! The screen is readable in a variety of lighting conditions.
- while on the subject of hardware, the keyboard feels nice. The one minor thing I have to get used to is the mousepad: occasionally, one of my thumbs accidentally touch it while I type, and the computer thinks I clicked on the mouse button.

Next step: installing Beryl! (Of course! That's the whole point of getting a laptop with Linux on it --to show off! :) )

User Journal

Journal: Dellbuntu laptop arrived! 1

Journal by KWTm

Today, the Dell laptop (Inspiron 1505n) with pre-installed Ubuntu arrived today. I will post my review blog as replies to my previous journal entry about Buying A Dellbuntu so that all the comments and threads can be organized into one place.

News flash: Nope, I won't be posting it to my previous journal entry because Slashdot has archived that entry and its replies, and no one can make changes. What the tutti fruitti!?? Okay, fine, I'll post the review as a reply to here.

User Journal

Journal: Buying a Dellbuntu 7

Journal by KWTm

I'm checking out the journal system on Slashdot. Hmm, it looks like I can enable and disable comments for individual journal entries. That's good.

Anyway, in this post about buying a Linux computer from Dell, I said I would post in my journal about how the purchase was going. Knowing me, I'll probably be too lazy to post much, but I figured that posting this in my journal would let others comment without having to post directly under the main thread.

Here's a copy of what I had posted:

I thought I should hang back and let others do the initial buying, to see how well this works out and whether the hardware crashes and burns. But if everyone did that, then nobody would buy because no one would want to be first. Since I've been looking forward to getting a Linux notebook, I think it should be okay for me to be one of the first "tryer-outers". Also, hopefully this venture of Dell's into Ubuntu will be high-profile enough that if I encounter any problems, I'll scream and shout that I'm going to post about my problems on Slashdot, and then Dell shall suffer the wrath of Slashdot!! and they'd be more willing to fix it.

In addition to the basic notebook at $599, I decided to upgrade the memory from 512MB to 2GB (+$200), since it's probably the most precious commodity around; if I try to upgrade later, say in 2 years, some new memory standard will probably have come out and I won't be able to find the proper chips.

I figured I'd upgrade the hard drive, too, from 80GB to 160GB. I had thought I would upgrade the 2.5" HDD myself, but it comes with a SATA hard drive, and I've only worked with PATA hard drives[1]. Anyway, that's another +$125 for the HDD upgrade.

My third upgrade is for the DVD burner. The original price comes with a CD burner/DVD-ROM drive, but I've always had problems with Linux and DVD burning --my Kubuntu box has the LITE-ON DVD DL burner, and so far I've had to power up our Win2k box to burn DVD's. For +$40, I'm happy to get the DVD DL burner, and I want to see if K3b will let me burn all 8GB+ onto a DL DVD. Would be sweet if I could.

The only thing I don't like is the screen size. I don't care about widescreen[2], and you can't directly compare diagonal screen sizes of 16:9 (widescreen) screens with 4:3 (conventional) sizes, so I converted. The diagonal of a 16:9 screen is 1.22 times as long as a 12:9 (that is, 4:3) screen for the same height, so I divided the 15.4" diagonal length of the widescreen by 1.22 to get 12.6". So I'm really getting a 12.6" screen, except it's wider. That's tiny. The ThinkPad that my work gives me is 15" (4:3 aspect, same screen height as 18.3" widescreen) and I don't think it's big enough. Well, at least the small screen size makes the laptop smaller and portable.

By the way, what the heck is "TrueLife (glossy)"? I have the option to have it or not have it for my screen, at the same price, but it sounds like a load of MarketSpeak.

So, anyway, here's my system, cut&pasted from the Dell page:

Intel® Pentium® dual-core proc T2080(1MB Cache/1.73GHz/533MHz FSB
Ubuntu Edition version 7.04
15.4 inch Wide Screen XGA Display with TrueLife(TM)(glossy)
2GB Shared Dual Channel DDR2 SDRAM at 533MHZ, 2 DIMM
160GB 5400 RPM SATA Hard Drive
8X CD/DVD Burner (DVD+/-RW) with double-layer DVD+R write capability

53 WHr 6-cell Lithium Ion Primary Battery
Intel PRO/Wireless 3945a/g

1Yr Ltd Warranty and Mail-In Service
Recycling Kit and Plant a Tree for Me

Intel® Graphics Media Accelerator 950
Integrated Audio
Intel Centrino Core Duo Processor

I'll probably sit on this till next week, and then make the purchase.
Any comments? Is this a good deal, or am I being foolish?

I'm experimenting with the Slashdot journal, so maybe I'll post stuff in my journal [slashdot.org] about how the purchase is going, and I think I can set it up so that people can post comments.

-----
[1] PATA notebook drives: It's not that I'm afraid of SATA drives; it's that I've been standardizing on PATA 2.5" drives because I have a number of 2.5" notebook enclosures that, for $25, turn the internal notebook HDD into an external USB HDD that fits into my shirt pocket.

[2] widescreen: Please don't give me that crap about "But if you're screen's not wide enough, you don't see the whole movie --it will be chopped off at the left and right sides!" Well, then, just shrink the movie! I don't see anyone ever saying, "You need a 4:3 screen, because your TV show will be chopped off at the top and bottom by a 16:9 screen!"

Google

Journal: Google's crappy domain registration

Journal by chimpo13

I decided to give a friend of mine a domain name for Xmas. Instead of using my normal server provider (pair.net), I decided to try out Google. What a mistake.

Google uses GoDaddy. I don't have a problem with GoDaddy. I simply wanted to forward the domain to her Blogspot blog. Blogspot is owned by Google. Google won't let me forward the domain. I talked to GoDaddy for an hour and 15 minutes (using up my cell phone minutes) while they tried their work arounds. Only, it's a Google owned domain. They pointed that out. It's not a domain that belongs to me -- it's Googles. And they weren't allowed to touch it. Of course, I had to talk to 4 different people for that. At least GoDaddy has great hold music.

They told me to talk to Google. Only, I can't get an answer from Google. Nothing but automated answers. I can't even cancel the domain through Google.

Eventually, I'll get this figured out, but it's been a pain in the ass. I think it's just been shifted over the those that are in charge of Orkut or one of the other failed Google ideas.

"Is it really you, Fuzz, or is it Memorex, or is it radiation sickness?" -- Sonic Disruptors comics

Working...