Components and connectors

Hi,

I am working on reverse-engineering a large-scale IT system, dozens of
independent software systems and possibly hundreds of interconnections
between them. The software systems are mostly third party software,
sometimes heavily customized, sometimes custom. This is far away from
the software level, much more into systems architecture.

I am looking for ways to describe the systems in a component-and-
connector style. The web seems to have little information on this,
books seem to be rare as well. Can you recommend or point to some best
practices in this area?

I am also interested in tools supporting this. I am looking for
something which allows for some textual representation plus a
graphical view, e.g. showing the connections for each component in a
list and annotating components and connectors with additional
descriptions.

I could do the textual representation in some editor or Excel and the
graphics in Visio but that would be disconnected. I would appreciate
to have some round-trip functionality. Any recommendations on tools?

Thanks,
Bernd

On Aug 11, 10:15 am, bernd.viefh...@web.de wrote:
> Hi,
>
> I am working on reverse-engineering a large-scale IT system, dozens of
> independent software systems and possibly hundreds of interconnections
> between them. The software systems are mostly third party software,
> sometimes heavily customized, sometimes custom. This is far away from
> the software level, much more into systems architecture.
>
> I am looking for ways to describe the systems in a component-and-
> connector style. The web seems to have little information on this,
> books seem to be rare as well. Can you recommend or point to some best
> practices in this area?
>
> I am also interested in tools supporting this. I am looking for
> something which allows for some textual representation plus a
> graphical view, e.g. showing the connections for each component in a
> list and annotating components and connectors with additional
> descriptions.
>
> I could do the textual representation in some editor or Excel and the
> graphics in Visio but that would be disconnected. I would appreciate
> to have some round-trip functionality. Any recommendations on tools?
>
> Thanks,
> Bernd

Perhaps something like a data flow diagram might work?

Documenting it in EXCEL is very helpful. (did something similar on a
project about 10 years ago.)

ed

Responding to bernd.viefhues...

You need to narrow the field down a bit. Is the design model you want
Structured or OO? Was the system developed with Structured or OO
techniques? Is the system well-formed for its construction paradigm?

> I am working on reverse-engineering a large-scale IT system, dozens of
> independent software systems and possibly hundreds of interconnections
> between them. The software systems are mostly third party software,
> sometimes heavily customized, sometimes custom. This is far away from
> the software level, much more into systems architecture.
>
> I am looking for ways to describe the systems in a component-and-
> connector style. The web seems to have little information on this,
> books seem to be rare as well. Can you recommend or point to some best
> practices in this area?

If the system was developed procedurally, you might find SysML useful
for this task because it emphasizes component-and-connector and has a
hierarchical structure.

> I am also interested in tools supporting this. I am looking for
> something which allows for some textual representation plus a
> graphical view, e.g. showing the connections for each component in a
> list and annotating components and connectors with additional
> descriptions.

If the development was in a common OOPL, there are a number of
commercial reverse engineering tools that will provide an OOD model of
the code. But it won't be connector-and-component.

Unfortunately the problem with reverse engineering tools applied to
large systems is that information about why design decisions were made
is lost at the 3GL level. So it is often impossible to separate
functional vs. nonfunctional requirements or OOA vs. OOD and OOD vs. OOP
decisions. Worse, the tool must make assumptions about what was intended
and what is design vs. tactical solution. Basically you end up with a
good view of the 3GL code but a pretty poor view of the design.

> I could do the textual representation in some editor or Excel and the
> graphics in Visio but that would be disconnected. I would appreciate
> to have some round-trip functionality. Any recommendations on tools?

Are you doing your own manual reverse engineering and just want to
record it? Then there are lots of drawing tools for both OO and
Structured approaches. They will usually provide plenty of connectivity
between bubbles & arrows and text descriptions. So I would recommend
getting a drawing tool for whatever methodology you use rather than
using VISIO. The good tools can provide things like syntax and
consistency checks.



--
There is nothing wrong with me that could
not be cured by a capful of Drano.

H. S. Lahman
hsl@pathfindermda.com
Pathfinder Solutions
http://www.pathfindermda.com
blog: http://pathfinderpeople.blogs.com/hslahman
"Model-Based Translation: The Next Step in Agile Development". Email
info@pathfindermda.com for your copy.
Pathfinder is hiring:
http://www.pathfindermda.com/about_us/careers_pos3.php.
(888)OOA-PATH

On 11 Aug., 21:23, "H. S. Lahman" <h...@pathfindermda.com> wrote:
> You need to narrow the field down a bit. Is the design model you want
> Structured or OO? Was the system developed with Structured or OO
> techniques? Is the system well-formed for its construction paradigm?

This is beyond the code level, therefore I would assume that
structured-vs-OO is not really applicable here. Some of my subsystems
are themselves large-scale systems, third-party software where I don't
have access to the codebase.

I am more or less looking at describing the system architecture as a
graph with nodes representing the various systems and edges showing
their interconnections.

> If the system was developed procedurally, you might find SysML useful
> for this task because it emphasizes component-and-connector and has a
> hierarchical structure.

SysML would be OK, as well as UML component diagrams.

> If the development was in a common OOPL, there are a number of
> commercial reverse engineering tools that will provide an OOD model of
> the code. But it won't be connector-and-component.

No code, no chance of reverse engineering.

> Are you doing your own manual reverse engineering and just want to
> record it?

Yes

> Then there are lots of drawing tools for both OO and
> Structured approaches. They will usually provide plenty of connectivity
> between bubbles & arrows and text descriptions. So I would recommend
> getting a drawing tool for whatever methodology you use rather than
> using VISIO. The good tools can provide things like syntax and
ors> consistency checks.

I could use one of the available UML editors, doing component
diagrams. However, so far I was not really pleased with the handling.
These tools tend to focus on the components and not non the
connectors. They usually lack transparency on the non-graphical
representation. Typically showing a tree structure one has an easily
accessible component catalog with interfaces per component. But it is
really hard to see for a given component with which other components
it is connected. In complex diagrams thats not easily visible due to
the number of elements.

Regards,
Bernd

Responding to bernd.viefhues...

Somehow I missed the fact that you are at the Systems Engineering level
rather than the Software Design level. Sorry about that.

>> You need to narrow the field down a bit. Is the design model you want
>> Structured or OO? Was the system developed with Structured or OO
>> techniques? Is the system well-formed for its construction paradigm?
>
> This is beyond the code level, therefore I would assume that
> structured-vs-OO is not really applicable here. Some of my subsystems
> are themselves large-scale systems, third-party software where I don't
> have access to the codebase.

It still makes a huge difference. Structured subsystems are defined
using pure functional decomposition while OO subsystems are based on
problem space subject matter invariants and level of abstraction. So the
architecture at the system partitioning level will usually look very
different.

This is potentially a very big problem when Systems Engineers use
Structured techniques (which are handy in hybrid systems where the
hardware is already described hierarchically via VHDL or whatever) but
the developers of the subsystem software use OO techniques. There will
often be substantial divergence concerning subsystem boundaries and
interfaces.

One way to look at this is that Structured functional decomposition is
only concerned with the customer functional requirements. But OO
development is used because an additional suite of nonfunctional
requirements comes into play: software maintainability.

In particular, the OO paradigm takes great pains to avoid hierarchical
flow of control because it was the root cause of most of the "spaghetti
code" from the Structured era. The dependencies in an OO component
Diagram at the subsystem level will look like a hierarchical graph but
that is purely superficial. The way subsystems are defined is very
different and "low level" subsystems are not extensions "high level"
subsystems that simply refine higher level requirements. (In fact, a
better characterization of "low level" OO subsystems is that they have
higher reuse potential across systems so they are necessarily standalone
modules.)

So let me push back by asking: why are you doing this? That is, is it
purely to document what's already there or will it be a basis for future
enhancements, replacing components, or other maintenance?

If it is purely for documentation, then I would use an approach that was
appropriate for the original design paradigm because that is the path of
least resistance. If it is to be a base for future maintenance, then the
approach should probably be consistent with whatever development
paradigm will be eventually used.

>> Then there are lots of drawing tools for both OO and
>> Structured approaches. They will usually provide plenty of connectivity
>> between bubbles & arrows and text descriptions. So I would recommend
>> getting a drawing tool for whatever methodology you use rather than
>> using VISIO. The good tools can provide things like syntax and
> ors> consistency checks.
>
> I could use one of the available UML editors, doing component
> diagrams. However, so far I was not really pleased with the handling.
> These tools tend to focus on the components and not non the
> connectors. They usually lack transparency on the non-graphical
> representation. Typically showing a tree structure one has an easily
> accessible component catalog with interfaces per component. But it is
> really hard to see for a given component with which other components
> it is connected. In complex diagrams thats not easily visible due to
> the number of elements.

The Component Diagram was defined assuming OOA/D construction
techniques. In Class, Package, and Component Diagrams connectivity is
defined, but the lines define logical connectivity or dependency among
entities rather than communications connectivity. One defines specific
collaboration communications in an Interaction Diagram.

Also, in OOA/D an entity and its interface are two different model
elements. In UML you can designate interfaces using an Interface
("lollipop") element or a Port element, both attached to the Component.
The actual message flows can then described on an Interaction Diagram
where the swimlanes reference the Components, Interfaces, and/or Ports.

I suspect Ports are close to what you want and they can be identified on
a Component Diagram. But they tend to work more naturally with SysML
blocks, which was mostly why they were introduced. (They are convenient
in that context because they can have an associated protocol state machine.)

In OOA/D there are really only three levels of decomposition: subsystem,
object, and object responsibility. (The OO paradigm uses level of
abstraction and a flexible notion of logical indivisibility to "flatten"
functional decomposition trees within these levels.) So the tree
structure the tool employs is designed to navigate that limited
hierarchy. IOW, it organizes the model elements conveniently but it
doesn't describe how they interact.

Finally, I would note that when the Component Diagram is used to define
systems at the subsystem level in the translation methodologies, the
connections between subsystems invariably also map communications.
That's because methodologically the diagram is a DAG where the
dependencies identify flows of requirements in a client/service manner.
In addition, there is a rigorous interface model for communications over
those connections and that model just happens to be very useful for
dealing with large scale reuse (e.g., third party components). So you
might find the category on "Application Partitioning" of interest in my
blog.


--
There is nothing wrong with me that could
not be cured by a capful of Drano.

H. S. Lahman
hsl@pathfindermda.com
Pathfinder Solutions
http://www.pathfindermda.com
blog: http://pathfinderpeople.blogs.com/hslahman
"Model-Based Translation: The Next Step in Agile Development". Email
info@pathfindermda.com for your copy.
Pathfinder is hiring:
http://www.pathfindermda.com/about_us/careers_pos3.php.
(888)OOA-PATH

On Aug 11, 9:15*am, bernd.viefh...@web.de wrote:
> Hi,
>
> I am working on reverse-engineering a large-scale IT system, dozens of
> independent software systems and possibly hundreds of interconnections
> between them. The software systems are mostly third party software,
> sometimes heavily customized, sometimes custom. This is far away from
> the software level, much more into systems architecture.
>
> I am looking for ways to describe the systems in a component-and-
> connector style. The web seems to have little information on this,
> books seem to be rare as well. Can you recommend or point to some best
> practices in this area?
>
> I am also interested in tools supporting this. I am looking for
> something which allows for some textual representation plus a
> graphical view, e.g. showing the connections for each component in a
> list and annotating components and connectors with additional
> descriptions.
>
> I could do the textual representation in some editor or Excel and the
> graphics in Visio but that would be disconnected. I would appreciate
> to have some round-trip functionality. Any recommendations on tools?
>
> Thanks,
> Bernd

Hi,
This really depends upon how formal your drawing needs to be and what
you are attempting to express. I recommend starting simply by drawing
a functional view of the system using a drawing tool such as Visio.
If there is a requirements document or design document this will help
you here.
Next, as another poster suggested, draw a data flow through the system
as it would occur with no errors. Follow this up by addressing the
alternate paths the data could take if an error occurs of if there is
more complex logic that routes the data through the system on
different paths. Again, if you don't need formality use Visio and
just a plain block flow.
Third, allocate the functions you identified to software and
hardware. It's okay if functions are allocated to more than one COTS
or custom component.
Fourth, the final view I would provide is the hardware, software and
network interrelationships. With the hardware include what software
is installed and if you want to go further link this to the functional-
software component mapping you did earlier.
I assume you have this system under CM so you should be able to pull
the hardware, software, and network components from here. Also if your
system has undergone a Functional Config Audit,use that information to
help define your functions.
What you are proposing to do I have done many times on systems that
were built in a short time with the intention of using them only until
a fully engineered solution could be provided...as often happens the
short term solution becomes the permanent solution for a variety of
reasons with none of the system engineering artifacts one would need
to effectively manage the system. Also, unless you are very familiar
with UML 2.0 and SysML which is an extension to the UML I would stay
away until you can bone up and practice on their use. Keep it simple
just to get your system documented then look to put it into a modeling
tool.
Good Luck.

bernd.viefhues@web.de wrote:

> Hi,
>
> I am working on reverse-engineering a large-scale IT system, dozens of
> independent software systems and possibly hundreds of interconnections
> between them. The software systems are mostly third party software,
> sometimes heavily customized, sometimes custom. This is far away from
> the software level, much more into systems architecture.
>
> I am looking for ways to describe the systems in a component-and-
> connector style. The web seems to have little information on this,
> books seem to be rare as well. Can you recommend or point to some best
....

Actually, this is Very Hard (aka "impossible") to do if the system was
constructed using "state of the art" techniques, which generally rely on
call-return to connect parts of the system together. Call-return (usually)
breaks causality, which defeats any attempt at sensible diagramming.

If, otoh, the system was constructed as a bunch of independent modules,
aka "processes" (using call-return only as a necessary evil instead of as a
fundamental design principle), then you do have a chance of diagramming it.

My thoughts on this subject are (briefly) described in the two papers at my
website

www.visualframeworksinc.com.

Basically, I treat all components as (very, very light-weight processes) and
send one-way "events" between them. We built a hybrid diagrammatic
programming language and used it in production applications.

You might also look at:

Paul Morrison's FBP: http://www.jpaulmorrison.com/fbp/

Ted Faison's book "Event-Based Programming - Taking events to the Limit"

I think that Ric Holt has done work on reverse engineering:
http://plg.uwaterloo.ca/~holt/

Google for the term "design recovery". Design recovery is a technique of
decomposing existing software into tiny factoids (e.g. prolog-like
assertions about relationships within a system) and then using inferencing
tools to traverse the factoids to "recover" interesting design intentions
from the pool of information. We used D.R. techniques to attack Y2K
problems. There were competitive technologies on the market at the time
(most used parse-based approaches instead of D.R.), but I don't remember
their names at the moment. (I.E. if you toy with various google phrases,
you might find a commercial tool that does some of the work for you,
e.g. "design recovery", "program understanding", "meta programming",
something along those lines).

pt