Mercury Quality Center - is there a worse solution? - Software-Testing
This is a discussion on Mercury Quality Center - is there a worse solution? - Software-Testing ; Vladimir Trushkin wrote:
> On Mar 2, 4:35 am, Michael Bolton <michael.a.bol...@gmail.com> wrote:
>
>>>>A couple of months before that though, I had written a very detailed
>>>>test plan for a program in Word and when we had to make ...
-
Re: Mercury Quality Center - is there a worse solution?
Vladimir Trushkin wrote:
> On Mar 2, 4:35 am, Michael Bolton <michael.a.bol...@gmail.com> wrote:
>
>>>>A couple of months before that though, I had written a very detailed
>>>>test plan for a program in Word and when we had to make another change
>>>>to that program, I simply attached the document and wrote in the step
>>>>something to the effect of "Follow all of the steps in the attached
>>>>document".
>>
>>>I would never do this. By doing so you have completely destroyed the
>>>intention implied with use of a tool. So, you better throw the tool
>>>away and start tracking test execution in Word. Yet, the latter is not
>>>the best choice either. If the sequence described by such a file is
>>>complex you have no visibility into test results.
>>
>>You have no visibility into the test results if (and only if)
>>
>>- the tester is incapable of explaining his actions; AND
>>- the tester has no memory of what he's done; AND
>>- the tester doesn't keep any notes; AND
>>- the application under test provides no log files; AND
>>- the tester has filed no problem reports (or any other report); AND
>>- the tester didn't use a program like BB Test Assistant, Camtasia, or
>>Spector to record her actions; AND
>>- there is no body of test data that the tester used; AND
>>- no other person communicated with the tester while she was testing;
>>AND
>>- you never debriefed the tester; AND
>>- you never observed or supervised the tester; AND
>>- you took no action to address any of the above problems.
>
>
> This might be true in some condition but it is not in line with my
> experience. IMO, you have no visibility into what was tested if your
> tests are not scripted (it does not take long to write steps)
Using a text editor, it doesn't, but using something like MQC, it really
does.
Come on - if you've used Windows for any length of time, you should be
used to such editing features as <Ctrl>-<Arrow> to navigate between words.
I can't tell you how many times I've done that and found myself 6 steps
back from the step I intended to work on - or maybe you're so perfect
that you never have to go back and edit what you just wrote.
> and if
> your tests are not scripted with enough details (test data, the way
> actions are taken).
Care to finish that thought? If they are not scripted with enough
details, then your test plan sucks whether you wrote it in MQC,
Microsoft Word or even in Crayon on construction paper.
> Taking notes is good and this is never a bad
> thing, but when you work with more than 1 tester you need them to be
> take consistently otherwise reading them will be a pain.
Ah - so apparently you are NOT familiar with proof-reading what you
wrote or editing it so it makes sense.
Reading anything in MQC IS a pain.
> If you need a
> form for the notes why can't this be a test case? Digging out logs for
> the piece of a useful information is not a good idea... I am sure you
> get my point 
How do you envision "digging out logs"?
And how is that any easier in MQC than it would be to dig out of an
organized set of directories in either Windows or Unix?
>
>
>>Otherwise, you doubtless have /some/ visibility into the test
>>results.
>>
>>
>>>If that big-bang step fails does it mean either 1 test failed or 100? and even if you can explain it right after the testing will you be able to do it a month later?
>>
>>These questions are ridiculous to anyone who bothers to think about
>>them for a minute, and who is willing to consider some alternative to
>>the conclusions that you've already drawn.
>>
>
>
> Ok. But when you look at 1 000 failed tests out of 10 000 trying to
> figure out what is wrong keeping tests atomic helps a lot.
The test where I attached a Word document is atomic - it's atomic at the
Word level.
Do you imagine people just click Fail at the step where the test plan
fails? (That is if MQC actually has a Fail button when you run the test).
Would they not say exactly what failed and what the results were?
> This is not
> a question of credibility (it may be though in some cases) but rather
> the question of being able to effectively manage test body.
>
Agreed - and if you want to effectively manage "test body", you would
not be using MQC.
Seriously - did Mercury even test Quality Center? Or were they victims
of themselves by relying on their own test "tool" to confirm that MQC
actually worked?
> ----
> Best Wishes,
> Vladimir
-
Re: Mercury Quality Center - is there a worse solution?
Our company has used MQC during the last 4 years, I have used it the
last year.
Very few use more then a bare minimum of the application (because its
such a pain to use).
No one wants to be educated to superuser.
In my experience MQC sucks. The question is what is the alternative
and how do you export all your testcases to an other tool????
-
Re: Mercury Quality Center - is there a worse solution?
daniel@interaktionsdesign.net wrote:
> Our company has used MQC during the last 4 years, I have used it the
> last year.
> Very few use more then a bare minimum of the application (because its
> such a pain to use).
> No one wants to be educated to superuser.
>
> In my experience MQC sucks. The question is what is the alternative
> and how do you export all your testcases to an other tool????
>
>
Well, the alternative happens to be well-motivated people who are
willing to grasp the entire process.
For example, I am currently investigating and rewriting a set of
programs which use shared memory and messge queues and ftp to
communicate amongst our own system and another system.
There is absolutely no documentation for this stuff and the comments in
the programs are laughable.
Serioously, I do not need a comment which tells me /* increment */ after
a statement like "some_variable++".
Were the people who wrote this shit originally for real?
Let's face it - they put comments like /* increment */ in the code just
so they could say they used comments. Forget the fact that the main
logic flow is not commented at all, let alone documented.
I'm looking for a paradigm shift because when I rewrite one of these
fucking programs, the test plan is going to involve about 7 other
programs and when it's done, that test plan will have successfully
tested the other programs as well.
Yet, with the way we are using MQC, there is no way to write a test for
a sub-system like this. We're supposed to write test plans for a single
program at a time.
No, it's not gonna work in this case and this will be the perfect
opportunity for me to escalate the issue of MQC sucking ass up to
management.
Instead of spending half a zillion dollars (or whatever MQC costs), we
should just hire a QA department even if that department consists of
only one person.
If the current situation persists, I can guarantee you that my
development process will consist of more rigorous testing than will ever
be done by relying on MQC.
---
Oh, and on a different note, an L1 visa-holder asked me what was wrong
with my test-plan yesterday. Well, I wrote this test-plan 6 months ago
and instead of investigating why a certain step was failing, he assumed
(wrongly) that the test plan was flawed. Well, maybe the test plan was
flawed since it did not catch this situation.
The fact of the matter was that there was unusual data which caused a
problem - he was re-using a test plan which had worked before, but whe
it was used this unusual data wasn't present.
Obvious solution - the test plan should have made sure this unusual data
was accounted for, but it wasn't. Mr. L1's reaction was to blame the
test plan and alter it since he wasn't getting the results he expected.
Naturally, I had to investigate it for him and discovered what was
REALLY wrong and told him that he would have to modify the code to
correct this bug.
In typical L1 fashion, he was resistant to this solution. I'm fucking
sick of immigrants too - sure, some of them are really sharp, but this
idiot gets paid 1/5 of what an American citizen is paid and is expected
to perform at the same level and he simply cannot. I wasted an hour of
my time yesterday doing his job for him while my own projects were
neglected.
There are legitimate uses of L1 visas and then there are people like him.
A legitimate use would be the programmers who came from the foreign
company that just bought us and are working to integrate their system
into our plants. An illegitimate use of L1 visas is to hire low-paid
foreigners who are not as competent as their American counterparts to do
the same thing for less money (and who don't do a good job either).
It's interesting that I can see both legitimate and illegitimate uses of
such visas among my co-workers.
-
Re: Mercury Quality Center - is there a worse solution?
> In typical L1 fashion, he was resistant to this solution. I'm fucking
> sick of immigrants too
I'm sure some "immigrants" are sick of "developers".
Your credibility takes a long, long walk when you argue this way.
---Michael B.
-
Re: Mercury Quality Center - is there a worse solution?
Michael Bolton wrote:
>>In typical L1 fashion, he was resistant to this solution. I'm fucking
>>sick of immigrants too
>
>
> I'm sure some "immigrants" are sick of "developers".
>
> Your credibility takes a long, long walk when you argue this way.
>
Well, point taken, but IME the L1 and H1-B visa holders who are worth
their salt are few and far between.
I don't mean to say that there aren't incompetent US citizens in this
field, but they're not as common as incompetent visa holders doing the
same job.
In fact, the guy I was complaining about in my last post is actually one
of the better ones, but if I were presented with the same problem, I
would take the bull by the horns and fix it rather than try to pawn it
off as an existing problem which wasn't relevant to the issue on the
radar at the moment.
But if a program does NOT pass QA, it does not pass QA. Don't try to
change the test plan unless the test plan is actually flawed! In this
case the only "flaw" in the test plan was that it was not comprehensive
enough and allowed this bug into production 6 months ago.
His solution was to lower the bar. To me that is not an option - I
wouldn't have even thought of doing that.
I wouldn't say things like I said in my earlier post if long-term
experience hadn't convinced me of it. Sure, I've seen incompetent and
lazy American citizens too, but the majority of us are not.
In working 20 years in IT, the American citizen who is incompetent or
lazy is the exception and gets fired (and deservedly so). The
visa-holder who is incompetent or lazy is the rule and when I meet one
of them who is neither of those things, I am impressed - and I know a
number of immigrants who I am impressed with. And yet, the incompetent
are usually tolerated - why? Because they are cheap, despite federal
law requiring that they be paid market wages.
So if you think I am trying to make the argument that people from China
or India or Russia or Europe or South America are incompentent, you are
the one who is bigoted. I don't care where anyone comes from, but the
fact that cheap labor is favored over competence is a very real problem.
I'd rather pay ONE competent worker - from anywhere - $500 an hour than
to pay 10 workers who did a half-assed job $50 an hour to do the same
project.
> ---Michael B.
-
Re: Mercury Quality Center - is there a worse solution?
Jumping into this thread ... bit late ..
Though I have no specific view points in support of or against Mercury
Quality center - MQC, GUI (I think this is what "developer" seems to
be complaining) ... I have a perspective to share about "Test
management systems" (MQC is a considered as a test management
software).
I recently worked with a Pharma client and based on the discussion I
have had with them - my view point on Test management systems changed
clearly. In regulated environment like Pharma, food and medical
devices - evidence of having tested superseeds everything - whether
you tested right thing, adquately and so on. I even heard statements
like "it is OK to wrong thing to please auditors" than not doing
anything or doing some right thing and not recording the things for
evidene.
So, let face some realities - in some contexts - systems like MQC are
like lifelines - without which nothing will hapen in those
environments. You might ask - why only MQC (one that sucks)? why not
others ? ... As I understand from conventions and practices in
regulated industry - even the chioce of the tools to collect and
preserve the evidence of having tested some stuff - also goes to
strict regulations and approvals. The reason why your management is
insisting on also could be something similar ....check that out ...
I sense that more than MQC - you seem to have concerns about how
testing (or developer testing to be precise) is to be conducted, what
are issues with developer testing her own code and so on .... these, I
belive are bigger problems and seem to cause for your frustration ...
Try having a pragmatic and open conversation with your manager and
suggest him about some alternatives to make your (other developers)
life easier and enjoyable ...
I would be happy to assist you with any "Testing" related questions
and concerns that you or your manger might have...
Shrini Kulkarni
Test Consultant
http://shrinik.blogspot.com
-
Re: Mercury Quality Center - is there a worse solution?
Shrinik wrote:
> Jumping into this thread ... bit late ..
>
> Though I have no specific view points in support of or against Mercury
> Quality center - MQC, GUI (I think this is what "developer" seems to
> be complaining) ... I have a perspective to share about "Test
> management systems" (MQC is a considered as a test management
> software).
>
> I recently worked with a Pharma client and based on the discussion I
> have had with them - my view point on Test management systems changed
> clearly. In regulated environment like Pharma, food and medical
> devices - evidence of having tested superseeds everything - whether
> you tested right thing, adquately and so on. I even heard statements
> like "it is OK to wrong thing to please auditors" than not doing
> anything or doing some right thing and not recording the things for
> evidene.
>
> So, let face some realities - in some contexts - systems like MQC are
> like lifelines - without which nothing will hapen in those
> environments. You might ask - why only MQC (one that sucks)? why not
> others ? ... As I understand from conventions and practices in
> regulated industry - even the chioce of the tools to collect and
> preserve the evidence of having tested some stuff - also goes to
> strict regulations and approvals. The reason why your management is
> insisting on also could be something similar ....check that out ...
>
> I sense that more than MQC - you seem to have concerns about how
> testing (or developer testing to be precise) is to be conducted, what
> are issues with developer testing her own code and so on .... these, I
> belive are bigger problems and seem to cause for your frustration ...
>
> Try having a pragmatic and open conversation with your manager and
> suggest him about some alternatives to make your (other developers)
> life easier and enjoyable ...
>
Unfortunately, in larger companies, managers don't always respond well
to common sense - hence we use MQC.
The issue really is that management doesn't trust developers to test
their own code, so if I do something as simple as adding an fprintf()
statement, I have to write a test plan which encompasses 60,000 lines of
code.
> I would be happy to assist you with any "Testing" related questions
> and concerns that you or your manger might have...
>
> Shrini Kulkarni
> Test Consultant
> http://shrinik.blogspot.com