Power of Ticketing

I am not sure whether this practice is followed across companies in industry. At least I am sure that few projects are not following this system.

We are aware of filing Bugs against the AUT (Application Under Test). Sometimes the issues filed in Bug tracking system all are not bugs, but some may found to be a Enhancement (Nice to have feature), some are simply 'Task' PR which is assigned against developers.

This Task PR (meaning that particular task has to be done by Assignee of the bug) can be very efficiently used by Test Manager to keep track of the tasks to be done by Testers.

Imagine in a testing module both Manual and Automation testers are working and at the beginning of the testing life cycle many things are discussed as 'to be done' and the way they are tracked is through emails, task requests in Outlook and simply by "Managers". We can use the bug tracking system for this purpose and Manager can raise 'Task' tickets against Testers for all the tasks he wanted tester to do. Or even tester can themselves can create and assign to themselves.

Examples for those task bugs are

(i) Creating new test cases for the new features (and getting sign off from product management and developers). This issue is considered as 'Completed' only when Peer Review/ Product management Review/Developers review/ Second level review is completed. We can create the workflow in our bug tracking system accordingly.

(ii) Select test cases from manual test case repository in order to automate and get sign off from Automation engineer.

(iii) Automate all the selected manual test cases in a feature and get Peer / Client sign off.

(iv) Finish self review for Performance appraisals.

(v) Verify all the bugs for this release.

(vi) Publish the Performance numbers between last release and this release

(vii) Clean the test cases (delete all the obsolete test cases) ..Criteria => Test cases written in past 3 years.

(viii) Complete the knowledge transfer session (this task issue is considered to be completed only if the person who is getting KT has given the reverse presentation and signing off the documents)

(ix) Do 5 interview before 30-Feb




and many more.

All these are treated as open Bugs and considered as important criteria in testing signoff of the particular release.

We can create separate areas in bug tracking systems, such as "Manual Testing Work" , "Automaton Testing work" etc This is analogous to IT help desk ticket but internal to testing team. How diligently we follow this reflect the success.

My 'Building' Experience

Few years back, I was working in a project as a single tester. That project was developed from scratch and everything is from India only. One of the additional responsibilities came to me is to 'build' the project through a tool call "Anthill". Since I was relatively new to industry I was excited and began to play with this. Since there was no proper planning,there were major fixes every day and I as a tester (who is having additional responsibility of 'building' with the latest fix) started taking builds on daily basis. In addition, for every bug I filed I started getting calls from programmers to 'check in the latest build..just to ensure..'.I was able to file bugs only after getting the latest code.

Point here is it will take at least one-two hours for me to take the latest build and if I find any showstopper, I stopped testing and started to wait for the fix and after the fix I again started 'building'.

There is no another tester and there is no change in this process, no comment from PM (since I didn't have test manager) and I was surrounded by developers.

After a while, I was finding only few bugs that too show stoppers and concentrating on improving the Anthill's build.xml to make quicker builds , which was lauded as good solution by few developers. In this, around 50% of my time went for running anthill / waiting for build / improving the anthill process.

All went fine until one morning when my client found one 'Critical bug' not 'show stopper' in one of the previous build (few days old), and obviously developers started saying to check in the latest build which he refused and filed the bug. The answer from him is albeit there may be some fix in the coming builds, this bug has been found in 'xyz' build and that is true and so I am filing the bug. Later we can change the status of the bug accordingly.

After this incident, I took the backseat and STOPPED building the code and made a build cycle. I made clearly that I myself as a tester will take the build every Wednesday and I will file bugs based on the build (irrespective of whether they are fixed in the later builds) and then I realized there were many Critical/Major serious bugs are there in the product which should have been caught long long back.

But my actions were taken at the end of the project and as expected our estimation went terribly wrong and blame came to me also as a tester (in fact I got a lion's share...:-( ).

Moral: (i) Decide / Negotiate about  the build cycle/ interval between builds at the early stage of the testing and follow the rule religiously (unless its very critical fix).
(ii) If you as a tester took additional responsibility of either build engineer, document writer , whistle blower in CMMi process or whatever, be clear to give priority to testing and then go for others.

OLE in Gmail

I am not sure whether this facility is available in Gmail or not, if it is not , it would be great to have.

I am not able to copy some excel fields and paste in my gmail. In other words, OLE is not supported in Gmail. I am not very sure whether OLE is possible outside Microsoft component.

If anybody has any work around for this (embedding an excel sheet/part of excel sheet in Gmail), I would be thankful :-)

Interesting Observation

Today i came across an interesting observation in my Yahoo Mail Inbox. I had some Birthday reminders which i did not read but deleted directly. Thereafter i opened an mail from one of my friends but the Birthday Alarm mail got opened. I was surprised and then again went back using the browser back button and again clicked on my friend's mail and once again the birthday alarm mail content got displayed. I refreshed again and tried and was able to see my friend's mail. Not sure what could have been the issue. Any such experience for readers?

Developers Reviewing test cases.

Recently I came across one process. Reviewing of test cases by developers. In my team testers are writing test cases (or scenarios) and getting reviewed them by developers, and the net result is developers who are finding test cases are 'suddenly' seeing new conditions, and remembering that they missed these conditions in their unit test or code. Hence, in my company code freeze date is about to change and net result is almost our test cases are passing , without much Bug filing.

Is this good practice? I definitely don't think so. Either the code freeze date should be rigid or Test case review shouldn't be done by developers. It should either inside test team or if it is needed product management should take care of this.

Cheating Tests

Another week goes by and more leading consumer websites crashed and burned, causing losses in both revenue (millions) and, perhaps more importantly, in consumer confidence in ecommerce. This week fail whales were sited swimming in the oceans of one of the world’s largest retailers, a leading children’s toy manufacturer and the industry’s leading online payment processor. PayPal alone (which was down for several hours this past Friday) may have lost hundreds of millions of dollars for itself and the enormous network of retail outlets that rely on them for their financial transaction services.

So why are fail whales continuing to happen so frequently? Don’t these companies test their sites? The answer, for load and performance testing, is of course they do…most of the time. In fact, some companies are spending millions of dollars on people, hardware and tools to load and performance test their websites. So why all the fail whales? One answer may be that organizations that have chosen the wrong testing tools or test service are, in fact, cheating on their tests!

Consumer facing website have become the primary channel of revenue and product information for millions of companies around the globe. So why cheat on testing? The pressure to deliver (business agility) is enormous today, for all IT organizations, and may well be a key reason cheating has become a common practice. Many test subcontractors and test companies cheat on their tests simply because they run out of time. According to PCWeek, this is what happened to AT&T on the pre-registration site for the iPhone4 launch. Due to a last minute feature upgrade they had no time to adequately performance test the site, which, of course, crashed an hour after it was launched, due to a 10x spike in traffic, creating a PR and revenue nightmare! Other organizations cheat because they just can’t afford the resources (people, hardware and tools) to properly test their sites in the first place.

The most common way to defend this cheating is to use semantics to obscure the shortcuts taken. When asked by an enraged business owner “did you test the site before going live?”, the answer is always “of course we did”. The problem is that it’s the wrong question. The right question is “did you test the site by accurately simulating real users performing both normal and unusual tasks, at and above expected volumes?”. For instance, if the goal was to simulate 5,000 concurrent users, a tester may respond that they tested for 5,000 “page views”! This is when language matters, since 5,000 page loads rarely equals the activity of 5,000 real users. In fact, it likely represents only a fraction of the target volume. By simply substituting page views or transactions for accurately simulating the activity of users on the site they almost certainly won’t reach the expected goal set by the business owner.

Another method of cheating the system is to adjust the timings of test scenarios. This practice is widely used by the testing community, primarily due to the cost of hardware and software when using traditional testing tools. For example, if buying a plane ticket online typically takes about 10 minutes, a clever tester may reduce the timing of this process in the test to just 1 minute. This, on the face of it, allows many more “users” into the system, but it doesn’t accurately simulate real world conditions. Finally, many leading edge companies are beginning to realize that the only way to accurately test a site is by including production testing. Testing only in the lab, and then extrapolating the results for the production environment, leaves far too many variables unaccounted for in the complex deployment environment that is the web. Again, cheating the testing system.

Whatever the reason for the cheating (lack of time, people, or resources) we must change this game now to maintain a high level of consumer confidence and continue to expand the growth of online commerce.

Source: www.soasta.com/blog/

Sales and Delivery

This post has nothing to do with testing. Just blogging my frustration.
Prospect calls company and asks to be connected to the sales team. The conversation is as follows:
Prospect: Hello, Can i speak to Mr.Xyz, VP sales?
VP Sales: Yes speaking, what can i do for you?
Prospect: We are interested in your IV & V services. Can you share your offerings?
VP Sales: We offer everything. Name it and you will have it.
Prospect: We have a testing requirement which is slightly complex but
(VP Sales interrupts)
VP Sales: There is nothing complex to us. We have 18000 man years of experience in testing.
Prospect: Ok, we need 2 resources at onsite for this requirement. The requirement is, we have 2 african lions suffering from digestion issues. We need a 2 testers(1 for each lion) to come open the lion's mouth some time after every meal and test if the food has digested properly.
VP Sales: Hmm..this is little challenging but i am sure we can do it. Hold on for a second. Let me pull in my Delivery Manager
VP Sales: So ABC, you are there right..ok..ABC is our Global Delivery Manager taking care of local operations. ABC, they have a simple requirement to test. I am sure we can do it. Can we talk about the rates?
ABC: What's the requirement?
VP Sales: Test if digestion is happening correctly..That's all
ABC: But how do we..
VP Sales: (Interrupts) You remember a similar case we did in Europe 2 years back..The customer was so happy with our "Blackbox Digestion Testing"
Prospect: Can you give us the reference?
VP Sales: No Problem. Once the contract gets signed, we'll give the reference.
Prospect: And can you get onsite resources? We also need substantial evidence on your capabilities.
VP Sales: Just a second..(Puts the line on mute). ABC, can we have 2 testers go to the nearby zoo tomorrow?
ABC: Yeah but why?
VP Sales: (Releases mute) To give you confidence what we can do is have a video shoot of the digestion process done by our testers which we can share over Webex or GoToMeeting. ABC has agreed to do it tomorrow evening itself. What would be your convenient time?
Prospect: Sounds like a plan. We can do it at 9 PST.
VP Sales: Allright cool, looks like we are all on the same page. Also, please note that the video shoot PoC is free of cost for you. Can we talk about the rates?
Prospect: Our current tester from the vendor side has lost 1 hand since he joined and now we have started paying them 2500$. We can start from 1500$ in your case.
VP Sales: I suggest that you don't pay us until we lose both the hands which in a way would prove the value of our services
Prospect: Excellent. Look forward to seeing the video shoot tomorrow.
VP Sales: I will in fact try my best to send it tonight. Convey my regards to the lions. I am sure they will enjoy our services. In case you have any questions or need more discounts anything, please let me know. Thanks. Oh sorry..forgot to tell you something. If you add more than 4 lions, we give a Project Manager free of cost as value add. Keep that in mind.
Prospect: Sure Ok. Have a great day.