sweltering in 39+ degree C (100F+) melbourne day
enjoing not having to go back to RCH with the young bloke
downloading old style video games '90's (cosmo, commander keen)
enjoying Indian Cricket team whip arrogant, cheating and old australian side.
wading in pool and feeding icypoles to young bloke and chick
waiting for cut price dev tools to arrive
should rtfm the
downloading Kings Quest for the young bloke and playing Hogs of War.
remember to update shipshape stuff. especially since I indexed the ratpile and the rest of the hard drive. created perl code to query all htm* and pdf files used template toolkit to output page and used ghostscript, ps2html and docset to generate pdf of printed page much like perl.apache.org
found why the delay in longhorn and possibly the journalling file system - dependency on yukon (pt 7 managing yukon dependencies). yukon is ms-sql server database in development.
update: 07DEC2003 - did not take too long. Check out SSC vs LinuxGazette.net Continued. Look out for ricks comments.
I'm always on the hunt for online books and references. One site I've thumbed through a lot is LinuxGazette. Filled with a broad range of Linux related topics, programming, sys admin etc.
Then doing some meta-moderation I came across this article, SSC Trademark Threats vs LinuxGazette.net, 12-02-03 9:05.
I originally became aware of this tussle after link harvesting www.linuxgazette.com for later use and checking out their author faq to see if it was worth writing an article or two. Now you know something fishy is going on when the shutters go down, then come back up again sans original crew plus heated boards.
But have ssc bit off more than they can chew? I'm a member of the Victorian Linux Users Group, LUV among others and so is Rick Moen. He actively pops up (even though he's on the flip side of the world) on the LUV mailing list answering technical questions etc, berating those who don't do their homework and helping others who do. I've never really found out why? but myself and others are grateful or at least amused. (I wish LUV's email list was online you could read the posts) The Age newspaper did article on him a while back (SSC used this as a publicity puff peice on their site for issue JAN2003 (see linux links).
stop sniffing my wallet!
In a world of creators and distributors, traditionally it's the distributors who make the cash. Go ahead make your money distributing CD's of documentation, printing Journals but don't take what is not yours - user generated documentation try to spin it into your own cashcow. You're not going to get any documentation from moi at your site.
whats a ternary conditional operator, '?:'.
I've been looking at *small* os systems for a while. They don't consume as much power are quiet and you can tuck them into small spaces.
Then I saw this article on daemonnews. Most users (non technical, phb's, grandmothers etc.) roll their eyes at motherboards, operating systems or science fiction especially if you can fit them into 1 sentence.
But if you tell them you have a *secure* internet connection over there in the corner that looks good (insert favourite bit of plastic cover) and they accept it as a *consumer grade* (read desirable) product. It's pretty shallow in some ways but aesthetics in consumer goods sell.
sent in entry to ask slashdot. was interested in what would happend to source code cultured with Lactobacillus. does the code still execute? how does the CD reader still read the code (but not alter it)?
After watching an intruiging ABC, Catalyst story Design by Fungus
about, Dr Cameron Jones
who accidently (in the great Australian tradition) spilt beer on a music CD
and played it
...use cells and atoms to perform useful computational tasks at the micron (10-6m) and/or nanoscales (10-9m) of organisation
... principally concerned with exploiting polymer and colloidal nano-agglomeration or biotechnology to control optical nonlinearity for new media applications ... M.M.P.
The applications of the fungal techniqes have been applied to (among other things) Visual design, audio and images.
I would like to ask budding molecular biologists/physicists what might actually happening here and for the programmers, "conjecture on what might happen to source code or binaries on a *fungal CD*".
experiment:
Aim:
Observe what happens to source code, object code or binaries when CD media they are contained on is covered with Lactobacillus bacteria.
method:
Cut some code to CD, or use an existing CD with binary code. (Be prepared to throw the CD away).
Apply bacillus to CD, culture bacullus and report results. Read
and view, *carefully* - (ie: clean
inner ring of CD in order for CD to be mounted) to see what
happens.
equipment:
The
bacteria suggested is Lactobacillus . any type of CD media, computer and source/object or binary code.
questions
It's a bit of a lame experiment as their are a lot of variables but has anyone had experience or willing to try this and write it up?
Had Ian Tuskett talking about DateTime modules in development at Melbourne PM last night. It was a pretty slick talk with clear overheads, examples and amusing anecdotes.
Tried some code out at home using Cygwin (DateTime installation failed), then ActiveState - also failed.
Here the ActiveState ppd collection status including DateTime - (red == failure, CPANRun "D" Builds Status (last updated Fri Nov 7 14:18:45 2003).
Another reason why ActiveState (read perl on windows) sucks. Time to ssh/telnet to my *nix boxes and try.
In commenting to this post on XBox/PS2 comparisons
back when ps2 first came out I ordered my console (within the first 10,000 machines in
It takes a while for developers to get used to the hardware, optimise the libraries but once they do the ability to translate game designers ideas into code ramps up.
It was a no brainer for developers who see a big market. This allows for a feedback loop to start. XBox has to start from scratch. With deep pockets and an interest in selling software lets wait for another couple of generations before we write it out of the console market.
just found an article about google and desktop searching using IE on the wires.
Google starts searching from the desktop
The Deskbar search software appears as an inset window on the Windows tool bar [computerworld]
Theres an faq and scant technical details at the news groups. But this post is asking pretty much exactly what I've beem working on with my code.
From: Names changed to protext the innocent
Subject: Operating System/File System search
This is the only article in this thread
View: Original Format
Newsgroups: google.public.labs.google-deskbar
Date: 2003-11-07 11:49:55 PSTI am hoping that someone at Google sees this (maybe it is already in
the works), and works on integrating search for local filesystems.
The new search utility in windows XP and 2003 is terrible, I can never
find files even when I know they are explicitly in a certain place on
my machine with a certain name or containing certain text.....
Looks like a fire and motion exercise by google and anyone even considering using this for a local file search replacement ought to read these articles.
Python has *the best* documentation out of the box. It really is up to date, useful and packed with usage examples. For the absolute latest in using Python it cannot be beat. This includes all the books, Learning Python, Programming Python [1.], etc... If you want to find out more on how to use Python download the documentation then the source code.
nb: I haven't read the Nutshell series so I can't comment. If you look at Mark's site you will see he is updating the *Learning Python* book so I also can't comment wrt to this title.
last updated: $Id: journal.html,v 1.2 2003/11/08 09:21:01 sah Exp $
optionparse:
Having had to parse options using your own code is a royal PIA.
Then I found optparse. Hidden in the Python [documentation] I found
how to use optparse [2.]. Sometimes I wish that the Python docs
had a key word cross referenced index because you really have to look to
find language keywords. The best place to look is the module index. Straight
out of the usage examples
type into your editor....
1 # test.py 2 from optparse import OptionParser 3 parser = OptionParser(usage = "usage: %prog 0.1 -f [-h], ") 4 parser.add_option("-f","--Foo",dest="foo_opt", help="this is foo 5 option") 6 (options,args) = parser.parse_args()
now you can try something like....
$python test.py -f bar
or
$python test.py -Foo bar
Best of all the help options are automagically supplied
$python test.py -h
usage: test.py 0.1 -s [-f -h], options: -h, --help show this help message and exit -fFOO, --foo=FOO this is foo option $
I used this technique for walk.py [3.], so to get the
options
$python walk.py --help
usage: test.py 0.1 -s [-f -h], options: -h, --help show this help message and exit -fFOO, --foo=FOO this is foo option $
I used this technique for walk.py [3.], so to get the
options
$python walk.py --help
usage: walk.py 1.5 -s [-a -o -D -r -h], <source filepath> options: --version show program's version number and exit -h, --help show this help message and exit -sSRC, --source=SRC read data from filepath -rRDF, --rdf=RDF read root, directory and file options -o, --out print data in rdf format to stdout -D, --Dictionary print data in dictionary format to stdout -aATRB, --Atrb=ATRB read file attributes, time, size, extension -tTPL, --tmp=TPL read template filepath for rdf markup
it makes the app easier to use without the burden of having to write your own code. This is common in python and I see it a lot at ActiveState. One person writes a tricky bit of *foo code* solving *bar problem* and someone else points out it's already in the language. this technique allows me to for instance pass in optional arguments to return different data depending on the data. Nothing out of the normal you may say, OptionParser just allows you develop faster.
e:\shipshape>d:\python23\python.exe python walk.py-ee:\shipshape
[e:\shipshape] {shipshape} ('decode.py') ('encode.py') ('export.py') ('metadata.py') ('metadata.py.bu') ('metadata.pyc') ('test.txt') ('walk.py') ('walk.py.bu') ('walkOLD.py') [e:\shipshape\CVS] {CVS} ('Entries') ('Entries.Extra') ('Repository') ('Root') [e:\shipshape\doc] {doc} ('.journal.txt.swp') ('index.html') ('index.html.bu') ('journal.txt') [e:\shipshape\doc\CVS] {CVS} ('Entries') ('Entries.Extra') ('Repository') ('Root') [e:\shipshape\meta] {meta} ('encode.mdf') ('walk.mdf') [e:\shipshape\meta\CVS] {CVS} ('Entries') ('Entries.Extra') ('Repository') ('Root') [e:\shipshape\sql] {sql} ('allHtml.cvs.sql') ('createTblDir.sql') ('createTblFile.sql') ('createTblRoot.sql') ('Fb.sql') ('reading.sql') ('selectAll.sql') ('selectAllRatpile.qry.sql') ('selectAllRatpile.sql') ('selectRatpile.sql') ('shipshape.sql') ('shipshape3.sql') ('shipshape4.sql') ('tools.sql') [e:\shipshape\sql\CVS] {CVS} ('Entries') ('Entries.Extra') ('Repository') ('Root') [e:\shipshape\sql\test] {test} ('selectAll.csv.txt') ('selectAll.sql') ('shipshape.sql') ('shipshape.wlk') [e:\shipshape\template] {template} ('encode.tpl') ('walk.tpl') ('walk.tpl.bu') [e:\shipshape\template\CVS] {CVS} ('Entries') ('Entries.Extra') ('Repository') ('Root')
then add more file detail using the -a option over the code source tree
e:\shipshape> e:\shipshape>d:\python23\python.exe walk.py-ee:\shipshape -a[1,1,1]
[e:\shipshape] {shipshape} ('.test.txt.swp',12288,'2003-11-08*05:19:16','swp') ('decode.py',3321,'2003-11-01*12:04:42','py') ('encode.py',5468,'2003-11-06*23:53:05','py') ('export.py',1682,'2003-11-01*12:04:42','py') ('metadata.py',9306,'2003-11-01*12:04:42','py') ('metadata.py.bu',9306,'2003-11-01*12:04:42','bu') ('metadata.pyc',13480,'2003-11-06*09:39:41','pyc') ('test.txt',0,'2003-11-08*04:51:46','txt') ('walk.py',9266,'2003-11-08*02:10:27','py') ('walk.py.bu',9266,'2003-11-08*02:10:27','bu') ('walkOLD.py',6922,'2003-11-07*08:44:13','py') [e:\shipshape\CVS] {CVS} ('Entries',266,'2003-11-08*02:10:27',NULL) ('Entries.Extra',121,'2003-11-08*02:10:27','Extra') ('Repository',16,'2003-11-01*12:04:42',NULL) ('Root',41,'2003-11-01*12:04:42',NULL) [e:\shipshape\doc] {doc} ('.journal.txt.swp',12288,'2003-11-08*04:58:56','swp') ('index.html',20701,'2003-11-02*23:15:15','html') ('index.html.bu',20701,'2003-11-02*07:37:18','bu') ('journal.txt',1708,'2003-11-08*04:58:56','txt') [e:\shipshape\doc\CVS] {CVS} ('Entries',47,'2003-11-02*23:15:15',NULL) ('Entries.Extra',16,'2003-11-02*23:15:15','Extra') ('Repository',20,'2003-11-02*13:08:54',NULL) ('Root',41,'2003-11-02*13:08:54',NULL) [e:\shipshape\meta] {meta} ('encode.mdf',706,'2003-11-01*12:04:42','mdf') ('walk.mdf',1078,'2003-11-07*21:52:17','mdf') [e:\shipshape\meta\CVS] {CVS} ('Entries',89,'2003-11-07*21:52:17',NULL) ('Entries.Extra',30,'2003-11-07*21:52:17','Extra') ('Repository',21,'2003-11-01*12:04:42',NULL) ('Root',41,'2003-11-01*12:04:42',NULL) [e:\shipshape\sql] {sql} ('allHtml.cvs.sql',658,'2003-11-02*06:38:56','sql') ('createTblDir.sql',610,'2003-11-01*12:18:18','sql') ('createTblFile.sql',614,'2003-11-01*12:18:18','sql') ('createTblRoot.sql',560,'2003-11-01*12:18:18','sql') ('Fb.sql',8084,'2003-11-02*06:19:26','sql') ('reading.sql',711649,'2003-11-02*06:23:03','sql') ('selectAll.sql',775,'2003-11-07*00:14:35','sql') ('selectAllRatpile.qry.sql',1131,'2003-11-06*22:29:13','sql') ('selectAllRatpile.sql',873,'2003-11-06*22:12:48','sql') ('selectRatpile.sql',800,'2003-11-03*08:07:33','sql') ('shipshape.sql',10757,'2003-11-02*04:31:51','sql') ('shipshape3.sql',6038,'2003-11-07*09:15:12','sql') ('shipshape4.sql',6764,'2003-11-07*09:38:53','sql') ('tools.sql',0,'2003-11-02*06:33:18','sql') [e:\shipshape\sql\CVS] {CVS} ('Entries',202,'2003-11-07*00:14:35',NULL) ('Entries.Extra',87,'2003-11-07*00:14:35','Extra') ('Repository',20,'2003-11-01*12:04:42',NULL) ('Root',41,'2003-11-01*12:04:42',NULL) [e:\shipshape\sql\test] {test} ('selectAll.csv.txt',1946,'2003-11-02*07:25:42','txt') ('selectAll.sql',339,'2003-11-02*07:25:21','sql') ('shipshape.sql',3536,'2003-11-02*07:26:24','sql') ('shipshape.wlk',771,'2003-11-02*07:27:36','wlk') [e:\shipshape\template] {template} ('encode.tpl',744,'2003-11-01*12:04:42','tpl') ('walk.tpl',388,'2003-11-07*21:51:43','tpl') ('walk.tpl.bu',388,'2003-11-07*21:49:51','bu') [e:\shipshape\template\CVS] {CVS} ('Entries',89,'2003-11-07*21:51:43',NULL) ('Entries.Extra',30,'2003-11-07*21:51:43','Extra') ('Repository',25,'2003-11-01*12:04:42',NULL) ('Root',41,'2003-11-01*12:04:42',NULL)
the -a option extracts file attributes, file size (bytes), file creation date (with some formatting) and file extension for every file. Having the option to turn some options on/off allows for more flexible code.
eval: parse plain-text data structures
if you look at the last optional arguement I passed
e:\shipshape> e:\shipshape>d:\python23\python.exe walk.py-ee:\shipshape -a[1,1,1]
Eval allows for Python to *parse and evalute* the text expression. Mark Lutz
explains in 'Learning Python' that
"... evaluates the specified expression (string or compiled code) in the optionally specified global and local namespaces... [4.Mark Lutz, Programming Python]"
There are problems, (performance, error handling, security) but these areas can be handled or reduced.
Plain text data structures + eval means you can parse in data structures
(pending complexity) and evaluate them
In designing an app trade-offs have to be made. This method could be seen as a *security* problem or a *user* problem but for me it hits the right level of power and leveraging existing language.
How much more work is involved creating my own templating language, then creating
a parser, validating the data, writing the documentation
References:
=============================================
1. Mark Lutz http://starship.python.net/crew/lutz/
2. Optparse - new in version 2.3 http://www.python.org/doc/2.3.2/lib/module-getopt.html
3. Read previous journal - generic cross platform tools to aid information management http://slashdot.org/comments.pl?sid=84421
4. Mark Lutz, David Ascher Programming Python, PP 222-224 Oreilly & Associates, 1999 First Edition.
http://www.oreilly.com/ ISBN: 1-56592-464-9
everythings shipshape sir...
Shipshape
with the email you read, the web pages you regularly visit,
the files you download...
name: Shipshape - everythings squared away and shipshape sir
date: 02NOV2003
programmer: goon
license: Authors rights have been asserted 2003 Not to be reproduced
without express permission.
Abstract
*Shipshape* is a set of programming tools I've *quickly* hacked together to
solve a particular problem of organising and working with files for any given file
paths on a file system. We shall attempt to create generic, cross-platform
set of tools to enhance the storage, manipulation and searching for
information normally stored on a file system
it within the fist page of searchs.
...To a computer they are all justs lists...
Discussion
The original problem revolved around having to maintain a html index page
of files. This is a simple problem solved by manually inserting the url links of
files within a file and maintaining them regularly using a text editor, cvs
and a bit of decision making.
Frankly doing this manually is a waste of time. Some files I read, delete - others I
would like to read but never quite get around to it. I either loose them or
simply forget where I have saved them.
Meanwhile the same problem presents itself with manual organisation of downloaded applications, files
that I suck down to read on regular intervals, organising my email lists.
This process is continually repeated for email, repeated for urls I visit,
repeated for files that I locate on my hard drive, ad nauseum.
Generic problem
This is a *generic problem*. You have the same problem happening with the
email you read, the web pages you regularly visit, the files you download.
To a computer they are all justs lists with attibutes, physical, logical, associative etc.
but how do we remember all of this? How can you search through this
information? There's got to be a better way.
This could explain why google is filling a gap in the web.
But I dont have my file system on the web. Google wont work on my machine.
Would I want my machine contents indexed?
Search for google article I knew I had somewhere in 124s. I didn't find the
precise article but enough to search the directory for the file I required.
# Connection: mysqlcctmp_1
# Host: 192.168.0.1
# Saved: 2003-11-02 23:29:24
#
SELECT
tblRoot.ID,
tblRoot.path AS ROOT,
tblRoot.pathRelative AS PATH,
tblDir.path as DIR,
tblFile.name as FILENAME
FROM
tblRoot
LEFT JOIN
tblDir
ON
tblRoot.ID = tblDir.rootID
LEFT JOIN
tblFile
ON
tblDir.ID = tblFile.dirID
WHERE
tblFile.name LIKE '%google%'
+-----+------+--------------------------------------+----------------------+---------------------+
| ID | ROOT | PATH | DIR | FILENAME |
+-----+------+--------------------------------------+----------------------+---------------------+
| 344 | e: | \reading\google | google | brokenGoogle.html |
| 344 | e: | \reading\google | google | google.html |
| 344 | e: | \reading\google | google | googlefeatures.html |
| 349 | e: | \reading\google\googlefeatures_files | googlefeatures_files | google.css |
| 349 | e: | \reading\google\googlefeatures_files | googlefeatures_files | google_sm.gif |
| 350 | e: | \reading\google\google_files | google_files | google.css |
| 350 | e: | \reading\google\google_files | google_files | google_sm.gif |
| 745 | e: | \reading\python\xml\webhack | webhack | pygoogle-0.5.3.zip |
| 745 | e: | \reading\python\xml\webhack | webhack | pygoogle-1.5.3.zip |
+-----+------+--------------------------------------+----------------------+---------------------+
Couldn't find it using mozillas book mark manager. It's intersting to note that had I searched for the directory 'google' and
assuming I had placed the files in such a directory I returned the results
back in 2.4s.
# Connection: mysqlcctmp_1
# Host: 192.168.0.1
# Saved: 2003-11-02 23:29:24
#
SELECT
tblRoot.ID,
tblRoot.path AS ROOT,
tblRoot.pathRelative AS PATH,
tblDir.path as DIR,
tblFile.name as FILENAME
FROM
tblRoot
LEFT JOIN
tblDir
ON
tblRoot.ID = tblDir.rootID
LEFT JOIN
tblFile
ON
tblDir.ID = tblFile.dirID
WHERE
tblDir.path = 'google'
+-----+------+-----------------+--------+---------------------+
| ID | ROOT | PATH | DIR | FILENAME |
+-----+------+-----------------+--------+---------------------+
| 344 | e: | \reading\google | google | bigbro.html |
| 344 | e: | \reading\google | google | broken.html |
| 344 | e: | \reading\google | google | brokenGoogle.html |
| 344 | e: | \reading\google | google | cookie.htm |
| 344 | e: | \reading\google | google | google.html |
| 344 | e: | \reading\google | google | googlefeatures.html |
| 344 | e: | \reading\google | google | jobad.html |
| 344 | e: | \reading\google | google | pagerank.html |
| 344 | e: | \reading\google | google | valerie.html |
+-----+------+-----------------+--------+---------------------+
Language
The language I chose for these first three steps is Python. For the language bigots it
could just as well of been Java, Perl, C# or (insert your own). Python has
the cross platform reach I require at the same time with an extensive standard
library.
... but how do we remember all of this? How can you search through this
information ?
Idea
The idea I've come up with is hardly new. Spider through a file path,
extract the useful information, massage it a bit and export it. I won't go
into great detail here but the simple technique I'm employing consists of:
Metadata: (metadata.py) useful system and information
that gives meaning to data collected.
Walk: (walk.py) meaning with a given root file url
recusivly scan and encode the directories and files within that
filepath. At the same time extracting important metadata.
Encode: (encode.py) read the *processed* walk data generated
previously, interpret and re-encode the data from a template. Again
including any important metadata.
Export: (export.py) with the encoded information export it.
Developed independently of each other and using stdin/stdout to read in and
export the data. It is planned for the walk, encode and export modules read their instuctions via template
files or commandline. The code can be reused from modules or used via the
command line using stdio. This allows great configurability.
At the present this test has not implemented the export module,
used templating for the options. Nor has the metadata module been used though
it has been constructed. The Encode module output was redirected to a file
and manually insterted into MySql then exported as a cvs file.
Here's the following example results.
1. Walk.py
The Walk module recursively reads the directory path in this case the
directory of the code - e:\shipshape.
The markup around the data is summarised as following:
[root]
root file path
{directory}
directory name
{file}
file name + extension
Running this code generates the following markup
d:> d:\python23\python.exe e:\shipshape\walk.py e:\shipshape
[e:\shipshape] {shipshape} (.encode.py.swp) (decode.py) (encode.py) (export.py) (metadata.py) (walk.py)
[e:\shipshape\CVS] {CVS} (Entries) (Entries.Extra) (Repository) (Root)
[e:\shipshape\meta] {meta} (encode.mdf)
[e:\shipshape\meta\CVS] {CVS} (Entries) (Entries.Extra) (Repository) (Root)
[e:\shipshape\sql] {sql} (allHtml.cvs.sql) (createTblDir.sql) (createTblFile.sql) (createTblRoot.sql) (Fb.sql) (reading.sql) (selectAll.sql) (shipshape.sql) (tools.sql)
[e:\shipshape\sql\CVS] {CVS} (Entries) (Entries.Extra) (Repository) (Root)
[e:\shipshape\sql\test] {test} (selectAll.csv.txt) (selectAll.sql) (shipshape.sql) (shipshape.wlk)
[e:\shipshape\template] {template} (encode.tpl)
[e:\shipshape\template\CVS] {CVS} (Entries) (Entries.Extra) (Repository) (Root)
2. encode.py
The encode module reads from stdin and interprets then encodes the
input with (at the moment) *hard coded* sql. The module will be made to read
from a template file. The encode module redirects the code to stdout.
Running this code generates the following markup
d:> d:\python23\python.exe e:\shipshape\walk.py e:\shipshape
| d:\python23\python.exe e:\shipshape\encode.py >
shipshape.sql
INSERT INTO tblRoot (tblRoot.path,tblRoot.pathRelative) VALUES("e:","\\shipshape");
INSERT INTO tblDir (tblDir.path, tblDir.rootID) VALUES
("shipshape",LAST_INSERT_ID());
INSERT INTO tblFile (tblFile.name, tblFile.dirID) VALUES
(".encode.py.swp",LAST_INSERT_ID()),
("decode.py",LAST_INSERT_ID()),
("encode.py",LAST_INSERT_ID()),
("encode.pyc",LAST_INSERT_ID()),
("export.py",LAST_INSERT_ID()),
("metadata.py",LAST_INSERT_ID()),
("metadata.pyc",LAST_INSERT_ID()),
("walk.py",LAST_INSERT_ID());
INSERT INTO tblRoot (tblRoot.path,tblRoot.pathRelative) VALUES("e:","\\shipshape\\CVS");
INSERT INTO tblDir (tblDir.path, tblDir.rootID) VALUES
("CVS",LAST_INSERT_ID());
INSERT INTO tblFile (tblFile.name, tblFile.dirID) VALUES
("Entries",LAST_INSERT_ID()),
("Entries.Extra",LAST_INSERT_ID()),
("Repository",LAST_INSERT_ID()),
("Root",LAST_INSERT_ID());
INSERT INTO tblRoot (tblRoot.path,tblRoot.pathRelative) VALUES("e:","\\shipshape\\meta");
INSERT INTO tblDir (tblDir.path, tblDir.rootID) VALUES
("meta",LAST_INSERT_ID());
INSERT INTO tblFile (tblFile.name, tblFile.dirID) VALUES
(("encode.mdf",LAST_INSERT_ID()));
INSERT INTO tblRoot (tblRoot.path,tblRoot.pathRelative) VALUES("e:","\\shipshape\\meta\\CVS");
INSERT INTO tblDir (tblDir.path, tblDir.rootID) VALUES
("CVS",LAST_INSERT_ID());
INSERT INTO tblFile (tblFile.name, tblFile.dirID) VALUES
("Entries",LAST_INSERT_ID()),
("Entries.Extra",LAST_INSERT_ID()),
("Repository",LAST_INSERT_ID()),
("Root",LAST_INSERT_ID());
INSERT INTO tblRoot (tblRoot.path,tblRoot.pathRelative) VALUES("e:","\\shipshape\\sql");
INSERT INTO tblDir (tblDir.path, tblDir.rootID) VALUES
("sql",LAST_INSERT_ID());
INSERT INTO tblFile (tblFile.name, tblFile.dirID) VALUES
("allHtml.cvs.sql",LAST_INSERT_ID()),
("createTblDir.sql",LAST_INSERT_ID()),
("createTblFile.sql",LAST_INSERT_ID()),
("createTblRoot.sql",LAST_INSERT_ID()),
("Fb.sql",LAST_INSERT_ID()),
("reading.sql",LAST_INSERT_ID()),
("selectAll.sql",LAST_INSERT_ID()),
("shipshape.sql",LAST_INSERT_ID()),
("tools.sql",LAST_INSERT_ID());
INSERT INTO tblRoot (tblRoot.path,tblRoot.pathRelative) VALUES("e:","\\shipshape\\sql\\CVS");
INSERT INTO tblDir (tblDir.path, tblDir.rootID) VALUES
("CVS",LAST_INSERT_ID());
INSERT INTO tblFile (tblFile.name, tblFile.dirID) VALUES
("Entries",LAST_INSERT_ID()),
("Entries.Extra",LAST_INSERT_ID()),
("Repository",LAST_INSERT_ID()),
("Root",LAST_INSERT_ID());
INSERT INTO tblRoot (tblRoot.path,tblRoot.pathRelative) VALUES("e:","\\shipshape\\sql\\test");
INSERT INTO tblDir (tblDir.path, tblDir.rootID) VALUES
("test",LAST_INSERT_ID());
INSERT INTO tblFile (tblFile.name, tblFile.dirID) VALUES
("selectAll.csv.txt",LAST_INSERT_ID()),
("selectAll.sql",LAST_INSERT_ID()),
("shipshape.sql",LAST_INSERT_ID()),
("shipshape.wlk",LAST_INSERT_ID());
INSERT INTO tblRoot (tblRoot.path,tblRoot.pathRelative) VALUES("e:","\\shipshape\\template");
INSERT INTO tblDir (tblDir.path, tblDir.rootID) VALUES
("template",LAST_INSERT_ID());
INSERT INTO tblFile (tblFile.name, tblFile.dirID) VALUES
(("encode.tpl",LAST_INSERT_ID()));
INSERT INTO tblRoot (tblRoot.path,tblRoot.pathRelative) VALUES("e:","\\shipshape\\template\\CVS");
INSERT INTO tblDir (tblDir.path, tblDir.rootID) VALUES
("CVS",LAST_INSERT_ID());
INSERT INTO tblFile (tblFile.name, tblFile.dirID) VALUES
("Entries",LAST_INSERT_ID()),
("Entries.Extra",LAST_INSERT_ID()),
("Repository",LAST_INSERT_ID()),
("Root",LAST_INSERT_ID());
3. mysql import/export
Code yet to be built, but trivial. Used the export tool to produce a csv of
the results.
# MySQLCC - [mysqlcctmp_1] Query Window
# Connection: mysqlcctmp_1
# Host: 192.168.0.1
# Saved: 2003-11-02 18:25:42
#
# Query:
# SELECT
# tblRoot.ID,
# tblRoot.path,
# tblRoot.pathRelative,
# tblDir.path,
# tblFile.name
# FROM
# tblRoot, tblDir, tblFile
# WHERE
# tblRoot.ID = tblDir.rootID AND
# tblFile.dirID = tblDir.ID AND
# tblRoot.pathRelative LIKE '%shipshape%'
#
'ID','path','pathRelative','path','name'
'1','e:','\shipshape','shipshape','.encode.py.swp'
'1','e:','\shipshape','shipshape','decode.py'
'1','e:','\shipshape','shipshape','encode.py'
'1','e:','\shipshape','shipshape','encode.py.bu'
'1','e:','\shipshape','shipshape','encode.pyc'
'1','e:','\shipshape','shipshape','export.py'
'1','e:','\shipshape','shipshape','metadata.py'
'1','e:','\shipshape','shipshape','metadata.pyc'
'1','e:','\shipshape','shipshape','walk.py'
'2','e:','\shipshape\CVS','CVS','Entries'
'2','e:','\shipshape\CVS','CVS','Entries.Extra'
'2','e:','\shipshape\CVS','CVS','Repository'
'2','e:','\shipshape\CVS','CVS','Root'
'4','e:','\shipshape\meta\CVS','CVS','Entries'
'4','e:','\shipshape\meta\CVS','CVS','Entries.Extra'
'4','e:','\shipshape\meta\CVS','CVS','Repository'
'4','e:','\shipshape\meta\CVS','CVS','Root'
'5','e:','\shipshape\sql','sql','createTblDir.sql'
'5','e:','\shipshape\sql','sql','createTblFile.sql'
'5','e:','\shipshape\sql','sql','createTblRoot.sql'
'5','e:','\shipshape\sql','sql','selectAll.sql'
'5','e:','\shipshape\sql','sql','shipshape.sql'
'6','e:','\shipshape\sql\CVS','CVS','Entries'
'6','e:','\shipshape\sql\CVS','CVS','Entries.Extra'
'6','e:','\shipshape\sql\CVS','CVS','Repository'
'6','e:','\shipshape\sql\CVS','CVS','Root'
'8','e:','\shipshape\template\CVS','CVS','Entries'
'8','e:','\shipshape\template\CVS','CVS','Entries.Extra'
'8','e:','\shipshape\template\CVS','CVS','Repository'
'8','e:','\shipshape\template\CVS','CVS','Root'
mysql table scripts
The table structure (at the moment) captures the following relationships,
Root => Dir => File.
The tblRoot table (at the moment) stores non
relational path information, path, pathrelative. eg: path - e: pathrelative - \shipshape\template.
#===
# name: createTblFile.sql
# date: 01NOV2003
# cvs: $Id: index.html,v 1.1 2003/11/02 13:07:39 sah Exp $
# programer: goon
# os written: windows
# space: tab
# sql vers: mysql specific, want 92
# description: sql to build tblFile
#
# bugs: none
#===
*/
# Host: 192.168.0.1
# Database: squared
# Table: 'tblroot'
#
CREATE TABLE `tblroot` (
`ID` int(11) NOT NULL auto_increment,
`path` varchar(100) NOT NULL default '',
`pathRelative` varchar(254) NOT NULL default '',
PRIMARY KEY (`ID`)
) TYPE=MyISAM;
The tblDir table (at the moment) stores the current directory path and
foreign key reference to tblRoot.
#===
# name: createTblDir.sql
# date: 01NOV2003
# cvs: $Id: index.html,v 1.1 2003/11/02 13:07:39 sah Exp $
# programer: goon
# os written: windows
# space: tab
# sql vers: mysql specific, want 92
# description: sql to build tblDir
#
# bugs: none
#===
*/
# Connection: mysqlcctmp_1
# Host: 192.168.0.1
# Saved: 2003-11-01 23:14:14
#
# Host: 192.168.0.1
# Database: squared
# Table: 'tbldir'
#
CREATE TABLE `tbldir` (
`ID` int(11) NOT NULL auto_increment,
`path` varchar(100) NOT NULL default '',
`rootID` int(11) NOT NULL default '0',
PRIMARY KEY (`ID`)
) TYPE=MyISAM;
The tblFile table (at the moment) stores
#===
# name: createTblFile.sql
# date: 01NOV2003
# cvs: $Id: index.html,v 1.1 2003/11/02 13:07:39 sah Exp $
# programer: goon
# os written: windows
# space: tab
# sql vers: mysql specific, want 92
# description: sql to build tblFile
#
# bugs: none
#===
*/
# Connection: mysqlcctmp_1
# Host: 192.168.0.1
# Saved: 2003-11-01 23:14:43
#
# Host: 192.168.0.1
# Database: squared
# Table: 'tblfile'
#
CREATE TABLE `tblfile` (
`ID` int(11) NOT NULL auto_increment,
`name` varchar(254) NOT NULL default '',
`dirID` int(11) NOT NULL default '0',
PRIMARY KEY (`ID`)
) TYPE=MyISAM;
Example searching for icon used
Heres a cheesy example. I need a python logo - python powered - would be good.
So I wrote the script, hit the query button MySqlCC and 120s
later I had the result.
Not exactly optimal but if I had a front end with a web server it would
allow me to see the result and use it.
# MySQLCC - [mysqlcctmp_1] Query Window
# Connection: mysqlcctmp_1
# Host: 192.168.0.1
# Saved: 2003-11-02 23:29:20
#
# Query:
# SELECT
# tblRoot.ID,
# tblRoot.path AS ROOT,
# tblRoot.pathRelative AS PATH,
# tblDir.path as DIR,
# tblFile.name as FILENAME
# FROM
# tblRoot
# LEFT JOIN
# tblDir
# ON
# tblRoot.ID = tblDir.rootID
# LEFT JOIN
# tblFile
# ON
# tblDir.ID = tblFile.dirID
# WHERE
# tblFile.name LIKE 'py%.gif' OR tblFile.name LIKE '%py%.jpg'
#
'ID','ROOT','PATH','DIR','FILENAME'
'296','e:','\reading\bored\organised\python\4199_files','4199_files','python2.gif'
'309','e:','\reading\bored\organised\spam\chinese_files\menu_data','menu_data','notpyth.jpg'
'681','e:','\reading\python\docpy23\icons','icons','pyfav.gif'
'686','e:','\reading\python\docpy23\misc\1690_files','1690_files','111-pythonnews.jpg'
'686','e:','\reading\python\docpy23\misc\1690_files','1690_files','python2.gif'
'688','e:','\reading\python\docpy23\misc\2388_files','2388_files','111-pythonnews.jpg'
'690','e:','\reading\python\docpy23\misc\2553_files','2553_files','python2.gif'
'700','e:','\reading\python\unorganised\culture_files','culture_files','PyBanner050.gif'
'700','e:','\reading\python\unorganised\culture_files','culture_files','PythonPoweredSmall.gif'
'702','e:','\reading\python\unorganised\process_files','process_files','PyBanner008.gif'
'702','e:','\reading\python\unorganised\process_files','process_files','PythonPoweredSmall.gif'
'703','e:','\reading\python\unorganised\tools_files','tools_files','PyBanner032.gif'
last updated: "$Id: index.html,v 1.1 2003/11/02 13:07:39 sah Exp $"
amazon supports online book searching using full text searchs. You can read more here at oreillynet. Log into amazon and search online - still try look up complex searchs like *openbsd i386 install low ram* and you dont get all books for instance OpenBSD : Implementing the Secure UNIX Platform. Not that useful. At the same time looks like they removed *book reading*.
Subject: problem 3.2 i386 install
From: peter renshaw
Date: Thu, 23 Oct 2003 22:39 +1000
To: misc@openbsd.org
Hi all,
I'm having fun installing OpenBsd 3.2 using an official CD [0] to a
low-end i386 [1]. Post installation the machine will not boot. The install
steps I've tried include
1. zero out HD/MBR
2. boot via floppy (no problems)
3. fdisk (error reported)
4. disklabel (error reported)
5. install software (error reported)
6. post install (correct error set boot blocks)
I've installed the 3.2 CD on higher end machines but this
particular machine is a *problem child*. I suspect bios is the problem.
(see hardware description [1]) Does anyone have any suggestions other
than get new HW or installing another os ?
Hardware, fdisk, disklabel, hexified boot & indicated
references, [#] below. This post has been mirrored [2].
Regs PR
P R E I N S T A L L
1. RTFM
-------
*FAQ, 4, 8 and 14 [3].
-4.3.1
-4.10.3
-8.9
-8.19 (yet to try)
-14
*man pages
-disklabel(8)
-fdisk(8)
-boot_i386(8)
-installboot(8)
-biosboot(8)
*cdrom docs
-INSTALL.chs
-INSTALL.i386
*google search
-various combos of keywords [4].
2. confirm HD and CHS
---------------------
*confirm cylinder, head, sector (chs) printed on hard drive (HD)
and on Western Digital (WD) website [5].
I N S T A L L S T E P S
1. zero out HD/MBR
------------------
*zeroed out HD (WD Lifeguard diagnostics, DLGDIAG for dos/win),
[6], [7].
*check supported hardware list [8].
2. boot via floppy (no problems)
--------------------------------
*boot via fd0 using floppy32.fs (CDROM will not boot)
-install method cd0
3. fdisk (error reported)
-------------------------
*select default, "do you want to use all of wd0 [No]"
-intention is to use entire disk for installation but avoid 4.10.3,
pt 2, "No valid boot loader was ever put on the disk".
*fdisk error reported [9].
-"fdisk sysctl (machinedep.bios.diskinfo) device not configured".
*reinit
*quit
*fdisk wd0 yields
Disk: wd0 geometry: 989/15/56 [830760 Sectors]
Offset: 0 Signature: 0xAA55
Starting Ending LBA Info:
#: id C H S - C H S [ start: size ]
----------------------------------------------------------------------
0: 00 0 0 0 - 0 0 0 [ 0: 0 ] unused
1: 00 0 0 0 - 0 0 0 [ 0: 0 ] unused
2: 00 0 0 0 - 0 0 0 [ 0: 0 ] unused
*3: A6 0 1 1 - 988 14 56 [ 56: 830704 ] OpenBSD
4. disklabel (error reported)
-----------------------------
*intention is very simple install first with 2 partitions, 1 root and
the other swap disk [10].
*disklabel reports error, "cant get disk bios device not configured".
*delete any existing partitions, leaving c.
*add "a" wd0a 800000 sectors [11].
*add "b" wd0b leftover sectors.
*quit saving change.
*disklabel wd0 yeilds
#
type: ESDI
disk: ESDI/IDE disk
label: WDC AC2420H
flags:
bytes/sector: 512
sectors/track: 56
tracks/cylinder: 15
sectors/cylinder: 840
cylinders: 989
total sectors: 830760
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0 # microseconds
track-to-track seek: 0 # microseconds
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 800000 56 4.2BSD 1024 8192 16 # (Cyl. 0*- 952*)
b: 30704 800056 swap # (Cyl. 952*- 988)
c: 830760 0 unused 0 0 # (Cyl. 0 - 988)
5. install software (error reported)
------------------------------------
*set hostname.
*do not config network (yet).
*set passwd
*install from cd0 sans x-server [12].
-[X] bsd
-[X] base32.tgz
-[X] etc32.tgz
-[X] misc32.tgz
-[X] comp32.tgz
-[X] man32.tgz
*set timezone
*make device nodes
*creating boot block displays following error [13].
-"unable to get BIOS geometry must specify -h -s"
6. post install (correct error, set boot blocks, uncertainty on path)
---------------------------------------------------------------------
*failure requires following from '14.8 - Installing Bootblocks'.
# cd
# cp boot
#
*is this right? I see many references [14] to
something like
# cp boot
#
*halt
*reboot
6. reboot (error reported)
--------------------------
*machine boots, following
Using drive 0 partition 3
reading boot
probing: pc0 com0 com1 mem[640k 15M a20=on]
disk: fd0
>> OpenBSD/i386 BOOT 1.29
Open(hd0a:/etc/boot.conf)
Cant read disklabel
*I interact
boot> machine diskinfo
Disk Bios# Type Cyls Heads Secs Flags Checksum
fd0 0x0 *none* 80 2 18 0x4 0x0
*and again
boot> machine boot hd0a
failed to read disk label
*hexify bootblock [15]
dmesg:
------
OpenBSD 3.2 (RAMDISK) #97: Thu Oct 3 20:06:10 MDT 2002
deraadt@i386.openbsd.org:/usr/src/sys/arch/i386/compile/RAMDISK
cpu0: Intel 486DX4 ("GenuineIntel" 486-class)
cpu0: FPU,V86
real mem = 16367616 (15984K)
avail mem = 11632640 (11360K)
using 225 buffers containing 921600 bytes (900K) of memory
mainbus0 (root)
bios0 at mainbus0: AT/286+(fd) BIOS, date 04/18/94
pcibios at bios0 function 0x1a not configured
bios0: ROM list: 0xc0000/0x8000 0xc8000/0x4000
isa0 at mainbus0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard
vga0 at isa0 port 0x3b0/48 iomem 0xa0000/131072
wsdisplay0 at vga0: console (80x25, vt100 emulation), using wskbd0
wdc0 at isa0 port 0x1f0/8 irq 14
wd0 at wdc0 channel 0 drive 0:
wd0: 16-sector PIO, CHS, 405MB, 989 cyl, 15 head, 56 sec, 830760
sectors
atapiscsi0 at wdc0 channel 0 drive 1
scsibus0 at atapiscsi0: 2 targets
cd0 at scsibus0 targ 0 lun 0: SCSI0 5/cdrom
removable
wd0(wdc0:0:0): using BIOS timings
cd0(wdc0:0:1): using BIOS timings
npx0 at isa0 port 0xf0/16: using exception 16
pccom0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
pccom1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
fd0 at fdc0 drive 0: 1.44MB 80 cyl, 2 head, 18 sec
biomask 4040 netmask 4040 ttymask 4042
rd0: fixed, 3560 blocks
root on rd0a
rootdev=0x1100 rrootdev=0x2f00 rawdev=0x2f02
fdisk:
------
Disk: wd0 geometry: 989/15/56 [830760 Sectors]
Offset: 0 Signature: 0xAA55
Starting Ending LBA Info:
#: id C H S - C H S [ start: size ]
----------------------------------------------------------------------
0: 00 0 0 0 - 0 0 0 [ 0: 0 ] unused
1: 00 0 0 0 - 0 0 0 [ 0: 0 ] unused
2: 00 0 0 0 - 0 0 0 [ 0: 0 ] unused
*3: A6 0 1 1 - 988 14 56 [ 56: 830704 ] OpenBSD
disklabel
---------
#
type: ESDI
disk: ESDI/IDE disk
label: WDC AC2420H
flags:
bytes/sector: 512
sectors/track: 56
tracks/cylinder: 15
sectors/cylinder: 840
cylinders: 989
total sectors: 830760
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0 # microseconds
track-to-track seek: 0 # microseconds
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 800000 56 4.2BSD 1024 8192 16 # (Cyl. 0*- 952*)
b: 30704 800056 swap # (Cyl. 952*- 988)
c: 830760 0 unused 0 0 # (Cyl. 0 - 988)
hexified bootblock
------------------
can be viewed at misc.openbsd.
notes:
======
[0] http://www.lsl.com.au/
[1] hardware:
cpu 486DX4
ram 16Mb
board VL/I-486SV2GX4
bios 04/18/94 SIS-85C471-I486SVG00 Award V4.05G
hd WDC A2C420H
chs 989/15/56
[2] mirrored at http://slashdot.org/~goon/journal and
http://www.netspace.net.au/~goonmail and
http://marc.theaimsgroup.com/?l=openbsd-misc
[3] ftp://ftp.openbsd.org/pub/OpenBSD/doc/obsd-faq.txt
[4] google searchs include
*machinedep.bios.diskinfo
*cant get disk bios device not configured
*openbsd 3.2 i386 install
*SIS-85C471 openbsd i386 install
*VL/I-486SV2GX4 openbsd i386 install
[5] http://www.wdc.com
[6] http://support.wdc.com/download/index.asp#windlg
[7] also used "dd if=/dev/zero of=/wd0 bs=512 count=1"
[8] INSTALL.i386 on 3.2 cdrom
[9] fdisk sysctl (machinedep.bios.diskinfo) device not configured.
[10] build PPPo gateway with firewall,
http://www.realo.ca/BSDinstall.html recommends using
simple setup with root and swap only.
[11] wd0a assign arbitrary 800000 sectors (391 Mb) to "/"
on final setup "/" will be == 373 Mb, wd0b swap == 32 Mb.
this is a naive setup for swap. The following post suggests that
"16Mb ram swapping in a *gateway/firewall system* is not likely"...
http://groups.google.com.au/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&th=3c43002ab8d1d43e&rnum=17
[12] comp32.tgz will b removed later for security reasons.
[13] this is covered in '4.10.3 - My i386 won't boot after install',
and 14.8 - Installing Bootblocks - i386 specific.
[14] installboot(8) and install faq, 4.8 - Installing Bootblocks - i386
specific.
# cd
# cd
[15] copy bootblock to disk and hexify using Vim and *hexman* by
Peter.Franz, http://vim.sourceforge.net/scripts/
# mount -t msdos
# dd if=/dev/rwd0c of=/mnt/mnt/wd0label bs=512 count=2
"$Id: problem_description,v 1.9 2003/10/23 12:22:45 sah Exp $"
"God is a comedian playing to an audience too afraid to laugh." - Voltaire
