Slashdot is powered by your submissions, so send in your scoop

 


Forgot your password?
Close
typodupeerror
×

Mapping/Understanding System Complexity? 63

Posted by Cliff from the threads-and-tapestries dept.
thesandbender asks: "I've recently inherited a project to 'simplify' the application environment for a company that has 1600+ service offerings (many of these are product 'foobar' that has options (like 'Alpha', 'Bravo', 'Charlie', and so forth) available. I am trying to map out the applications' dependencies from a technological and a business standpoint. I would like to designate a group of applications as depending on concepts, technologies (like SAN, DB2 and AIX), specific customers (like 'Bravo' and 'Charlie') and legacy applications. Basically, I want to define any number of arbitrary dependencies and then be able to map them out in a graphical format. With those maps I can show the business oriented staff how removing one application will affect other applications, and I can show the technically oriented staff how removing one system will affect other systems or applications. Has anyone in the Slashdot community run across such a tool? If you haven't, have you run across the need for such a tool? What would you want from it so that I can fashion a usable tool that addresses everyone's needs and not just my own?"
"The most appropriate tool-sets I've found to date are 'mind mapping' or 'concept mapping' tools. All of the tools I've found so far only allow me to create any number of ideas or concepts and don't allow for arbitrary, searchable and/or mappable attributes (e.g. Application 'foo' maps to attributes 'SAN', 'Java', 'Solaris' and 'Buy-Side') that would allow me to create hard and soft groupings that were based on defined attributes (e.g. I could ask for a cloud of all objects that share a specific technical attribute, and another cloud of objects that share a specific business attribute)."
This discussion has been archived. No new comments can be posted.

Mapping/Understanding System Complexity? More

Mapping/Understanding System Complexity?

Comments Filter:
  • I'd use a database modelling tool, like ERWin or Dezign for Databases. (Yes, they're both commercial - I've not seen a single good entity-relationship modelling tool that is also Open Source, although it's an obvious tool to write.)


    These tools map attributes in records to other attributes in other records. They're designed to then turn these maps into SQL code, but that part isn't important here. What is important is that you can create a full relationship mapping between entities. If you then treat the direction of the relationship as showing the dependency, you can map all the dependencies in the system.


    Managers like diagrams to be of a format that are familiar to them, so anything that is "better" from a technical standpoint but "less familiar" to managers from an experience standpoint is, in fact, not as good of a solution.

  • Graphviz (Score:5, Informative)

    by danpat ( 119101 ) on Thursday July 27, 2006 @08:07PM (#15795700) Homepage
    You can probably draw the picture you want with GraphViz, found here http://www.graphviz.org/ [graphviz.org]

    To use it, you create a text file that defines all your dependencies, it'll look something like this:

        digraph thingies {
            "app1" -> "SAN";
            "app1" -> "Java";
            "client1" -> "app1"; ...
        }

    You can then go on to group things together so they show up in meaningful locations on the diagram,
    associate pictures with certain nodes, put labels on things, make things in colour, etc.

    GraphViz takes care of the laying out parts (where best to put nodes and edges). Sometimes it takes
    a while to define everything in format that gets draw neatly, the results are often impressive and
    very useful.

    On coming to a new job, I've used it to draw all the dependencies between a collection of a couple
    of hundred SQL stored procedures in our database. The locals were horrified to have what they all
    knew in their gut depicted to them on 35 A3 sheets of paper on a wall :-) It was quite useful for
    identifying things recursive calls ("Oh, *thats* why that proc sometimes never ends...").
    • Nifty. Here's someone using graphviz for a similar situation - dependencies for apps in the FreeBSD ports system [cs.ru.nl].

    • I'm the author and I appreciate you taking the time to point this application out.

      Thankfully I have access to 48x60 plotters and I know I'll be using them :)

      • No worries. If you have access to a Mac, I'd also recommend http://www.pixelglow.com/graphviz/ [pixelglow.com]. The raster formats that
        can come out of GraphViz can be huge (i.e. 100,000x100,000 pixels), they can be quite difficult to deal
        with sometimes, the GUI these guys have written can deal with even huge graphs pretty snappily.

    • I've recently been using Graphviz with gprof and gprof2dot.awk to map out some really crufty old code. It's really illuminating.

      However, what I'd really like to be able to do is have it animated, showing nodes and edges in the graph as they become activated (slowed down from real time, of course and reducing super-repetitive actions logarithmically).

      A web application for this kind of graph layout and animation using SVG would be widely applicable, not just to visualizing call trees for computer programs,

    • I did this exact same thing with an application with 132 database tables, connecting them by foreign_key and so on. It didn't really give me a much better grasp on it than I already had, though. It's just a lot of information to digest (mine was only 20 pages - maybe I needed to make it bigger).
  • I saw a demo of this product and it seemed neat. You install the daemons on your systems and it monitors all the socket & file opens. In this way it can map application dependencies on different machines or the same machine. I think it can scale to a few thousand machines. On the down side it's not free. When Symantec acquired Relicore the product was renamed to some bland, information free name like Configuration Manager. http://www.symantec.com/Products/enterprise?c=prod info&refId=1461 [symantec.com]
    • This seems like it wouldn't be that hard to reproduce the core functionality. Obviously the management friendly graphs, reports, and the config management portion are more work but simply writing a tool that shows you what your box actually does sounds rather straight forward. And it would also seem that the 90/10 rule would come into play, in that you'll get most of the benefit by simply getting the raw data and doing some basic analysis on it without needing to spend all that money.

      Thanks for pointing thi

  • I recomend one that handles "backlinks" (ie: what are the pages that point to this page?)

    Using a wiki, I would add one page per Application, describing it and linking to its neighbor applications. I would also add one page for each attribute (like: Java, or Windows, or something like that), one page for each software group (might be related to the one before) and one page for each type of category (for instance: platform, application, development group).

    You can even use the hability of some wikis to do

    • I second this. I'm in the process of switching to another company and one thing I never had time to do at the current place is document. It was always a low priority because we were always going.

      I installed a wiki for the helpdesk and decided to create another one for the SA team. The linking alone has helped us tremendously. I've also created a system profile template wiki entry that can be copied and pasted and used by operators to help us document the systems.

      Let me know if anyone wants a copy of the tem

  • "Complexity kills" (Score:2, Interesting)

    by Baldrson ( 78598 ) *
    This request brings to mind the now famous quote from Ray Ozzie regarding reorienting Microsoft to services: "Complexity kills".

    There is actually a rigorous definition of complexity that is actually stated in terms of software: Kolmogorov complexity. Kolmogorov complexity is the size of the shortest program that produces, precisely, a given output. The number of bits in that program is the Kolmogorov complexity of the output. This is actually a very useful way of viewing system complexity in the case o
    • Outstanding! This is an Ultramax post! I had completely forgotten about old Andrey.

      I just have a problem with the architect of Lotus Notes proclaiming: "Complexity kills".

    • The problem with the concept of complexity based on program length, is that program length is highly dependent on the system of encoding used to represent the program. Short programs in functional languages can be quite long in imperative languages, and lamdba calculus functions for even simple things are sometimes so long as to be impossible to represent in a published paper.

      I've yet to see any "cannnonical" representation that can be used for this purpose. "Kolmogorov complexity" is not useful for the
      • It turns out that the choice of universal machine is not very important to the compression ratio. It is different by a constant -- which you can see by virtue of the fact that U1 can emulate universal machine U2 by a fixed sized program.
      • For small programs, yes.

        But for large programs, the size of the encoding is not significant, and decreases to irrelevant with increasing program size.

        Put simply, if you have a really efficient encoding ("A") of some program, you can use it to represent the program in any other encoding that's required ("B") by using your efficient encoding as data, input to an interpreter for that encoding (you might also view the interpreter as a decompressor).

        You mention lambda calculus as something that's way too long a
    • > As it happens -- unsurprising to many of us -- it is now a theorem of computer science that the closer the size of that "executable" gets to the Kolmogorov complexity of the use cases, the better. This theorem is a major breakthrough in CS and should be learned in every institute purporting to teach IT.

      Could you give us a source on that theorem, please?

  • Simple (Score:5, Funny)

    by jlarocco ( 851450 ) on Thursday July 27, 2006 @09:10PM (#15795931) Homepage

    When I get an assignment like this, I try to take a proactive stance. First, I add the project to my action item list. Then I formulate a list of stakeholders in the project. Then I call a meeting to open a dialog between the stakeholders and myself. After drilling down and making sure we're all on the same page, I draft a scope document. When I'm satisfied with the scope document, I hold a sidebar meeting to touch base with the shareholders and verify the document meets their requirements.

    Usually by that time the project gets assigned to someone else.

  • On the topic of mind mapping software, I always take a copy of Freemind [sourceforge.net] with me wherever I go. It's simple, quick and open source.
    • Thanks for posting the link to Freemind. I hadn't heard of "mind mapping" per se but this particular technique may help with some current projects.

  • Visio (Score:1)

    by p!ssa ( 660270 ) *
    Visio has always met my needs, you can "wow" the bus. types with the right stencil sets. Researching and identifying the catagories and dependancies will be the hard part regardless of the tool used. If you have additional people (preferrably experts with the various systems) I would define your key catagories, systems, packages etc. and assign the work of detialing the breakout tasks to each subject matter expert. Once you have this set of tasks completed bring everyone together to map the dependancies a
    • I'm sorry, but for complex things, Visio is just what you don't need. Sure it's nice and fancy on the output, but when you're trying to define a whole process it gets clogged and difficult on the input or later to change something.

      I think the user is looking more for a simple programming language or layout specification (like you do in TeX) in which he can write his stuff easily and orderly. Then the output gets automagically and dynamically generated and all he needs to do is make it look nice (for manager
  • A better tool than GraphViz for this application is Cytoscape - http://cytoscape.org/ [cytoscape.org] It was originally designed for biological applications, but is a general network visualizer with a very simple input format e.g.

    ModelA-Charlie dependsOn ModelQ
    ModelQ requires Java
    etc.

    You can visualize, interact with and edit the resulting network and even do some advanced analyses, such as network clustering which would tell you which families of projects you have.

    It is written in Java and is LGPL.
  • If it were me, I would redefine the goal slightly. It is not just the dependencies. It is the behaviour of the overall system, the collection of application components, that you need to be able to explain. In fact, if you want to know which components you can remove or tweak, you actually need to understand the business processes they are supposed to be supporting. Theoretically, the raw business processes of each company will represent less complexity than the system of application components you need to
  • Don't focus on tools too much, one options ids to get hold of some IBM documentation on "The Method". This gives you a great framework tp model complex systems in in such a way that is is actually useful.

    beware though, it's a method, not a bible, use from it what works for you.

    Can't help you to links, find a buddy who works there.
  • This is a fairly obvious application of graph visualization tools. Unfortunately, since the topic is so difficult, all the good solutions tend to be commercial. GraphViz is nice up to a point, but is pretty crummy when you get into thousands of nodes and edges. JGraph is another open source tool, and it's reasonable good, but as with other free software, it starts to choke on larger graphs.

    Some of the better commercial packages are from Tom Sawyer, AiSee, Ilog and yWorks.
  • The problem you are trying to solve is one of the areas addressed by the ITIL. In this particular case, mapping dependencies, you might want to look at a CMDB tool. There are a bunch of vendors in the space, and I'm guessing that you don't want to draw this diagram and forget it, but would actually like to have the information stay around for a while and be up to date. Your applications, hardware, customers, technologies, etc. are business entities. The relationships between them are what you are current
    • Correct, if you have some spare time you could also give a look at the sw we have build to deal with ITIL in our organization (Comune di Udine - City Council of Udine) http://www.cmdbuild.org/ [cmdbuild.org] (sorry, the site use only italian lang) The sw is open source (GPL), web based, written in Java, uses Postgresql as database
  • What you're doing is a form of network analysis, and there are programs that will assist with this, but there is also a learning curve. First, while there are other network analysis programs, I am familiar with social network analysis tools, so I will recommend two here. There are probably other slashdotters who can recommend other programs. I've als given a reference to a book that will get you going, but any math text on graph theory will do just as well.
    Programs:

    Agna (Benta, I. (2002, 2003). Agna Proj

  • I suggest two ways: A high level "bizniz map" and a detailed "data trail".
    My company is a polyglot conglomerated transmogrification of several gov't contracts, divisions of previous contractors whose contracts were consolidated, organizational divisons both on gov't and corp sides. We had the same problem as budget cuts forced us to compress.
    The data trail identifies sets of data from their origin points thru every app and person that touches the data, making special note of decision points. A decision p
  • Disclaimer: This is exactly what my company does, so I am by no means impartial...

    This is called application portfolio management (APM). As other people have said in the thread, it basically involves mapping the applications and their dependencies onto a graph. Then you can use the graph to build reports, perform searches, visualize the structure, etc. The company I work for (http://www.metallect.com/ [metallect.com]) does exactly this, and the website has much more info about it.
  • I've used info mapping techniques for a long time when I want to get my head around complexity - or even start thinking about a new project for the first time. There are dozens of software implementations. Here are some I know of:
    3D Topicscape; Aviz Thought Mapper; Axon Idea Processor; BrainMine; Claro Concepts; Concept X7; ConceptDraw Mindmap; Conception & InterModeller; Freemind (SourceForge); Headcase; Hypersoft-net; I-Navigation; InfoPro (ZPAY); InfoRapid KnowledgeMap 2005e; Inspiration Software

Slashdot Top Deals

1 + 1 = 3, for large values of 1.

Close