APL Program Code Metrics

Recently, one of my clients started using Visual Studio 2008, and one
of the new features is something called Code Metrics. In short, this
feature generates a statistic, a "maintainability index" (MI) for
every program.

The formula is something like

MI := 171
- (5.2 * ln(halstead volume))
- (0.23 * cyclical complexity)
- (16.2 * ln(lines of code))

This looks as if it would work for APL code with control structures.
(Cyclical complexity is basically a depth count of nested loops, if
statements, case statements, and so on) (How would one do this
meaningfully for legacy branching?)

Has anyone ever done anything like this for APL functions?

On 11 Jun, 13:48, Ibeam2000 <Ibeam2...@gmail.com> wrote:
> Recently, one of my clients started using Visual Studio 2008, and one
> of the new features is something called Code Metrics. In short, this
> feature generates a statistic, a "maintainability index" (MI) for
> every program.
>
> The formula is something like
>
> MI := 171
> - (5.2 * ln(halstead volume))
> - (0.23 * cyclical complexity)
> - (16.2 * ln(lines of code))
>
...
> Has anyone ever done anything like this for APL functions?

Hmm, I would be very sceptical of anything like this, especially if a
manager uses it as a measure of the quality of a programmer's work.

To take some obvious examples of why this is unlikely to be of any
use:

- Two lines of code can be clearer and more maintainable than one.
This is especially true in APL, but to a lesser extent is true in
scalar languages as well. It all depends.

- Depth of nested loops is usually determined by the complexity of the
problem, not of the solution. Sure, in some cases in might be more
readable to break it up so that inner loops are placed in separate
functions, but in some cases it might not. It all depends.

- These kinds of metric incentivise the programmer to write lots of
very small functions, rather than fewer larger ones. But that's not
always a good thing. I've quite often been frustrated in trying to
understand C or C++ code because it is broken up into pieces which are
too small - you have to act as a human C compiler to follow the logic
(You're trying to understand a loop. It calls a function, so you save
your 'understanding stack' and go and look at that function. It does
nothing significant and then calls another function, so you save your
'understanding stack' again, and so on. Finally you come to a tiny
piece of actual code which could have been placed in the original
loop, where it would have been obvious what it did. Meanwhile you've
forgotten the original context.) So, again, it all depends.

Basically, there is no substitute for intelligent human judgement in
this. Some people write very clear and maintainable code, and some
don't. The style can vary from one good programmer to another good
programmer. In general, those who think clearly write clear code.

But if you DO want a metric, in my experience there's only one worth
having. I propose, free of charge and free of copyright, Nabavi's
Code Quality Metric, which applies to any language. It is C divided
by L, where:

- C is the number of lines with comments, excluding any boiler-plate
headers etc which can be pasted in from a template or generated
automatically;
- L is the number of code lines.

Anything over 1 is good for most languages. You may want a higher
value for APL, where one line of code can do so much.

Even this metric should be used with care, however, as it is only
valid if the comments are well thought-out, and explain what the
programmer thinks the program is doing. This is because, if you're
trying to find a bug in someone else's code, the main question you are
asking is: Does the code actually do, under all conditions, what the
programmer thought it does?

Richard Nabavi
MicroAPL Ltd

BTW this is Microsoft, right? The same people who produce a grammar-
checker (in Word) which is complete garbage? Try it on well-known
pieces of extremely clear and well-written English, such as
Churchill's speeches, and you'll see what I mean...

I think software metrics could be useful, but in well managed and
tightly controlled situations. For me, I need a rough idea of which
functions might need some attention.

The context where I would find this kind of metric userful is in a
maintenance situation when one is just given the task to maintain a
system containing of lots (thousands) of functions. I don't have
enough time to go through every function in detail. The code quality
can range from good to dismal to indescribably ghastly. The one
useful statistic, where the errors occur, is naturally missing. After
eliminating dead code (often code volume drops 20% or more), I now
need to know which functions to pre-emptively rewrite. If a function
is both "ghastly" and "problematic", I rewrite it, or more likely,
clean it up. Or "ghastly" and "slow", etc.

It's like a three-legged stool. Maintainability is only one dimension
of the problem. More important are reliability and speed.

I also feel this paricular metric, as it might apply for C#, is rather
naive for APL. Spaghetti code detection isn't exactly cyclomatic
complexity. Creation and use of global variables is simply not
addressed, and bad style does not necessarily make the code more or
less maintainable.

On Jun 11, 1:48 pm, Ibeam2000 <Ibeam2...@gmail.com> wrote:
> Recently, one of my clients started using Visual Studio 2008, and one
> of the new features is something called Code Metrics. In short, this
> feature generates a statistic, a "maintainability index" (MI) for
> every program.
>
> The formula is something like
>
> MI := 171
> - (5.2 * ln(halstead volume))
> - (0.23 * cyclical complexity)
> - (16.2 * ln(lines of code))
>
> This looks as if it would work for APL code with control structures.
> (Cyclical complexity is basically a depth count of nested loops, if
> statements, case statements, and so on) (How would one do this
> meaningfully for legacy branching?)
>
> Has anyone ever done anything like this for APL functions?

I wrote something years ago which did attempt to generate these
metrics. The intention was to give guidelines to the developer on the
"opinion" of the process on his/her code. It wasn't really meant for a
manager who would use it out of content.

As Richard says, it's entirely dependent on how you write your
solution to the problem - is the purpose of a piece of code a "once
off" (although those tend to hang around much longer than you expect -
"The service life of a cobbled up fix is inversely proportional to the
time required to slap it together." [Nick Lappos]) or is it a critical
part of a system which runs 24 hours a day in multiple locations? The
"Quality Priorities" which change vastly with different circumstances.

Having said that it IS possible to gather these metrics (the first
version was in APL*Plus for DOS back in the late 80s or early 90s), in
that we certainly achieved consistent results which we (generally)
agreed with - but we never showed the results to anyone outside of the
team :-)

Chris Hogan

On 12 Jun, 15:24, crishog <goo...@4xtra.com> wrote:
>
> Having said that it IS possible to gather these metrics (the first
> version was in APL*Plus for DOS back in the late 80s or early 90s), in
> that we certainly achieved consistent results which we (generally)
> agreed with - but we never showed the results to anyone outside of the
> team :-)
>

I think that if you are trying to encourage (or force!) a particular
style for a particular type of software within a smallish team, then
such a metric makes more sense. But that's not the same as claiming
that the metric is some sort of objective measure of code quality,
programmer skill, or code maintainability, over a wide range of
different situations.

Richard Nabavi
MicroAPL Ltd

On Jun 12, 3:38 pm, micro...@microapl.demon.co.uk wrote:
> On 12 Jun, 15:24, crishog <goo...@4xtra.com> wrote:
>

> I think that if you are trying to encourage (or force!) a particular
> style for a particular type of software within a smallish team, then
> such a metric makes more sense. But that's not the same as claiming
> that the metric is some sort of objective measure of code quality,
> programmer skill, or code maintainability, over a wide range of
> different situations.
>
> Richard Nabavi
> MicroAPL Ltd

Lord no! I never try to force APLers to do anything: as John Jacob
says it's "like trying to herd cats".

All we were doing was trying to foster a house style across a team of
about 8-10, to make it a bit easier to jump into someone else's shoes
in an emergency. I don't think my code had anything like the rigour,
or even the sample size, to be able to call it objective.

We found it useful, because it provided a mechanism for evaluation. We
all look at other people code (and I hope our own) & think "that looks
good/bad". The metrics functions gave an impartial view, one effect of
which was more often to say the result was good enough, stop polishing
the code because you feel it's not neat enough & get on with the next
job. The reverse of "it's not good enough do it again".

Chris

On Jun 13, 10:29 am, crishog <goo...@4xtra.com> wrote:
> On Jun 12, 3:38 pm, micro...@microapl.demon.co.uk wrote:
>
> > On 12 Jun, 15:24, crishog <goo...@4xtra.com> wrote:
>
> > I think that if you are trying to encourage (or force!) a particular
> > style for a particular type of software within a smallish team, then
> > such a metric makes more sense. But that's not the same as claiming
> > that the metric is some sort of objective measure of code quality,
> > programmer skill, or code maintainability, over a wide range of
> > different situations.
>
> > Richard Nabavi
> > MicroAPL Ltd
>
> Lord no! I never try to force APLers to do anything: as John Jacob
> says it's "like trying to herd cats".
>
> All we were doing was trying to foster a house style across a team of
> about 8-10, to make it a bit easier to jump into someone else's shoes
> in an emergency. I don't think my code had anything like the rigour,
> or even the sample size, to be able to call it objective.
>
> We found it useful, because it provided a mechanism for evaluation. We
> all look at other people code (and I hope our own) & think "that looks
> good/bad". The metrics functions gave an impartial view, one effect of
> which was more often to say the result was good enough, stop polishing
> the code because you feel it's not neat enough & get on with the next
> job. The reverse of "it's not good enough do it again".
>
> Chris

This thread encourages me in my view that writing software is more
like writing than it is like engineering.

"Program Code Metrics" sounds a reasonable subject for engineers to
study. Contemplate an equivalent for, say, novelists, and the eyes
pop.

This is not to argue that how one writes – that is, one's style of
writing – is any less important in writing software than in writing
novels. But it might discourage us from thinking it any less deep or
subtle a subject either.

Metaphors matter. This is another example of when it's more helpful to
think of ourselves as writers (describing imaginary machines) than
engineers.

Simple rules have their value, as they do in teaching English
composition to young children. But they will only carry one so far.

This is not to argue there is nothing to be said about style, only
that it is unlikely to be simple or categorical, any more than it is
for EngLit students.

I don't know anyone in our field who thinks writing style matters as
much to the lifetime cost of software as I think it does. (And no –
wry grin – I _don't_ have metrics for that.) But, just as in writing
English, I have never seen a rule it hasn't been profitable and
appropriate to break on occasion.

Persig, I think, got it about right in "Zen and the Art of Motorcycle
Maintenance". He remembers teaching English when younger. Plagued by
student demands for clear rules, he works his way to the conclusion
that Quality is irreducible. It can be studied, but not reduced to
anything else in the sense, for example, that chemistry reduces to
physics.

So paradoxical conclusion: writing style matters very much to
professional writers concerned with the cost and serviceability of
software, and merits deep study. For most purposes (think Hemingway,
not Henry James) good style looks disarmingly simple. But there is
little simple to be said about it.

Stephen Taylor
editor@vector.org.uk
http://www.vector.org.uk/
http://aplwiki.aplteam.com/

There are things like the Flesch-Kincaid index which purport to give a
readability index for English text, though what it would make of the
difference between Hemingway and Dickens, let alone Shakespeare, I
honestly don't know.

I quote one comment from a site which advocates their usage: "A
readability index score is not an exact science. For example, it does
not consider disposition (paragraphs) or actual content (are the words
from a specific domain?)." - Which sounds like the reservations
expressed here.

Now don't get me wrong. I think "Software Engineering" is fairly much
an oxymoron. It implies some solidity and consistency to the process
of writing software: Like building a bridge - and we know what
happened to the millennium bridge in London. Many managers are failed
developers: they don't really understand the art of programming & they
feel the need to have some sort of handle or measurement of what these
magicians are up to.

My brother is an engineer - a mechanical engineer that is. The
impression I get of how he works though is more of an artist hand-
crafting something rather than it being a coherent science. He says
that often, even when following all the rules precisely, they don't
know if a machine will work until it's been built. Hence the
recognized need for prototyping, at least in mechanical engineering:
and never get him started on the problems of taking a working device
measured in American units & trying to built the same thing in
Imperial or Metric measurements.

However, as Lord Kelvin said “If you cannot measure it, you cannot
improve it.” and undoubtedly the Flesch-Kincaid index does measure
some aspects of written English and Halstead et. al. do measure
something about code. Readability does aid maintainability, but so do
internal comments and, best of all, well written code. None of the
program metrics make one a better programmer, nor are they perfect
tools, but (Kelvin again) "The more you understand what is wrong with
a figure, the more valuable that figure becomes."

But I'd not thought before of extending the literary analogy: some
authors are, or were, masters of short stories (Poe for example),
others playwrights, or novelists. Perhaps the different styles of
developer should be recognized in this way. We all know someone who is
a better system programmer, or a support person or an applications
programmer, and this must of necessity be reflected in the coding
styles they use.

I'm beginning to ramble.

Chris

On 20 Jun, 13:56, crishog <goo...@4xtra.com> wrote:
>
> But I'd not thought before of extending the literary analogy: some
> authors are, or were, masters of short stories (Poe for example),
> others playwrights, or novelists. Perhaps the different styles of
> developer should be recognized in this way. We all know someone who is
> a better system programmer, or a support person or an applications
> programmer, and this must of necessity be reflected in the coding
> styles they use.
>

I think that Stephen's point about programming being more like writing
than engineering is astute. Both Jane Austen and Hemingway wrote
excellent English, but I don't think any computer-calculated metric
would give both of them high marks.

The analogy goes further. To write clearly, you need to think
clearly, and understand your subject. I believe the same is true of
writing software. The actual style may vary, but clarity is all.
Programming 'bugs' are very often like poorly-written English
sentences; they arise because the author has not considered that the
text might be misconstrued.

What worries me is that trying to measure software quality by using
metrics is as absurd as saying that this:

"Five score years ago, a great American, in whose symbolic shadow we
stand today, signed the Emancipation Proclamation. This momentous
decree came as a great beacon light of hope to millions of Negro
slaves who had been seared in the flames of withering injustice. It
came as a joyous daybreak to end the long night of their captivity.
But one hundred years later, the Negro still is not free. One hundred
years later, the life of the Negro is still sadly crippled by the
manacles of segregation and the chains of discrimination."

is inferior to this:

"Five score years ago, a great American, in whose symbolic shadow we
stand today, signed the Emancipation Proclamation. This momentous
decree came as a great beacon light of hope to millions of Negro
slaves who the flames of withering injustice had seared. It came as a
joyous daybreak to end the long night of their captivity.
However, one hundred years later, the Negro still is not free. One
hundred years later, the manacles of segregation and the chains of
discrimination still sadly cripple the life of the Negro."

(The first is Martin Luther King's original text from the 'I have a
dream' speech. The second is what he would have said if he'd had to
put it through Microsoft's grammer checker.)

Richard Nabavi
MicroAPL Ltd

A question for Pirsig (Yen and the art of APL Maintenance): What is
the opposite of Quality?

It is precisely this negative quality I would like to estimate and
track and measure about code. It's not about programmer quality, nor
programmer productivity, nor exploring why the code got to this
state. If anything, it's about labour relations.

Suppose I have a group, three or more, humans who are working on some
code base. Two people working together perhaps know each other and
their respective coding habits rather well. But with three or more,
invariably the individuals become more disconnected. All too often,
one hears the phrase "This code is $#@% !"

But, as a sort of social contract, if I can suggest to try keep the
FOO index up above (some arbitrary number), and, in our copious free
time, and armed with other metrics, such as frequency of use, examine
and clean up functions which score poorly, I would expect the
following to happen:

Expectations
[1] Better "teamwork", at least some reduced tension and stress
amongst the group members
[2] Irrespective of what the FOO index indicates, an increase in code
quality, which leads to;
[3] An increase in code maintainability, which leads to;
[4] An increase in code reliability, and finally;
[5] goto 1

> “If you cannot measure it, you cannot improve it.”
> "The more you understand what is wrong with a figure, the more valuable that figure becomes."

These are excellent points. The metric can and should be developed by
the group, and it should be constantly refined (tinkered with). But
most importantly, the metric should be a catalyst for change, and not
an absolute measure of anything.