Thursday, March 13, 2008

Are you on uTest... I am....Check out the interesting things happening...

Boston-based uTest has raised $1.7M to bring to market a crowd-sourced quality assurance
(QA) marketplace and community. The service has begun recruiting testers in anticipation of its official launch, expected in early 2008.
No fuzzy logic here; uTest presents clear and communicable value propositions: For companies in need of QA, uTest is providing an on-demand environment for the management of full testing cycles. For testers, uTest is providing an oDesk-like marketplace through which they can be hired and paid on a Pay-per-Bug basis.
The company’s founders recognized that QA and usability testing solutions are inefficient. QA departments are either under-utilized when waiting for versions to test, or over-extended when a new version comes out. To most SMBs, QA outsourcing is neither cost effective nor available “on-demand”. Then, of course, there are cost-cutting measures that leave companies unable to sufficiently fund QA efforts, ultimately shipping buggy applications to users like us.
The uTest testing platform itself is entirely Web-based and provides the management of complete QA cycles, from creating/loading test scripts, to selecting the target community (profile and environment) of testers required. The platform also provides test case management tools, real-time information on bugs and defects logged, statistical information on release maturity level, as well as QA coverage and market readiness. Support for bug tracking systems includes Bugzilla, Jira, FogBugs and will be expanded over time.

In a smart business move, uTest will not charge companies to use their platform to manage testing and QA cycles—this will be entirely free of charge. uTest will only charge for the services provided by the community of testers.
The uTest offering makes a lot of sense to me and I expect it will be warmly embraced for several reasons. First, consider for a moment that for many SMBs, QA management solutions such as those offered by Mercury or IBM-Rational are beyond their reach. Speaking from personal experience, the majority of the testing cycles I’ve seen have been “managed” on Word or Excel. The “on-demand” model which has been proven time and time again on the Web from CRM to ERP, should work just as well for QA. The fact that the platform will be offered free of charge, pushes the offering to “no-brainer land”.
Second, beyond the manageability aspects of uTest’s service, companies will obviously enjoy the cost saving aspects of paying testers by the bug. It’s this aspect that in my opinion will make uTest’s offering relevant not only to SMBs but to large organizations as well. Logic would dictate that if they embraced off-shoring and near-shoring, crowd-sourced QA shouldn’t be too jagged of a pill to swallow.
Third, recruiting a userbase of testers should not be difficult. There are droves of potential testers in countries such as India, China, Russia, Bulgaria, Estonia, etc. Also, getting hired through sites like oDesk, Elance and RentACoder is becoming increasingly difficult due to the growing number of service providers. These same individuals can theoretically provide testing services instead of programming.
All-in-all uTest sounds quite promising and has earned a place on my companies to watch in 2008 list.

Saturday, January 19, 2008

Confused IT Engineers

I was reading this very nice article about this famous fable "Grass is always green on the other side of the fence". It reminded me of an conversation which i had with a IT Engineer some time back. He wanted to move from his present company due to few reasons which gives an insight into confused IT engineers mind. Somehow i feel like that the same confusing situation is present in every Software Engineer's mind or rather the phenomenon is more common in entry level candidates as they don't know what is that which they exactly wanted to do. Some get fascinated by the Higher salaries, some by onsite opportunities and some just want to jump without proper analysis about the type of work which the Organization where they are going to join will be offering them. I feel that this situation is more prevelant among the group of people who are in early Years of experience levels say people with 1-3 years experience. Here goes the conversation:

IT Engineer: What is the promotion cycle in Company X? Do they give any weightage to previous experience?
Madhukar: Since u have around 1.5 years exp only earlier hence u might get prmotion to senior software engineer at least after 2 years but that all always depends on your performance. It might happen earlier as well. If you had around 4 years to 5 years of prior exp then they would have taken you mostly as senior engineer, but not now.

IT Engineer: 2 years after I join them or of total experience?
Madhukar: 2 years after u join them would be an ideal time, however it might happen earlier as well.

IT Engineer: ok
IT Engineer: and do they have salary blocks for Engg and senior Engg?
Madhukar: no..average hike will be 10-15% or 12-16% every year like other companies normally give but this as well depends on the performance.

IT Engineer: ok and what is the componenet of there salary like basic pay, felxible benefits and Specail compensation plan etc.
Madhukar: lta, basic, bonus, gratuity etc.

IT Engineer now stops asking questions. I thought why not i ask him some questions just to know his opinions.

Madhukar: What difference does it makes if you are not hired as a Senior Software Engineer at Company X...?Means what is your take on this?
IT Engineer: Right now I am working as Senior Executive Quality and if I stay with this compnay longer I would become lead executive by end of this year so If I decide and go to Company X ... then I would have to start as fresher and then after minimum two years I will become what I am right now.

Madhukar: Ok why do you want to change your present job when u feel u have so much opportunities at the current company?
IT Engineer: tell you honestlyI was thinking of leaving my organisation 3-4 month back

Madhukar: hmm ok...

IT Engineer: and now this came to me like a surprise because i was not expecting anything like that now and here in my company in last 3- 4 month things have improved a lot.

Madhukar: So what are your priorities and long term goals?
IT Engineer: I learnt a lot in this company and still I am learning and if i stay here for another 3-4 months I would get the oppurtunuty to impement CMMi in my company.

Madhukar: What CMM level does Company X has?
IT Engineer: they dont go for CMMi, they say they have there own standards.

Madhukar: Then what is the compelling factor which says you to change since in this case you will be moving to a place where you will not be able to work on your interest area. Lets say a company Hires you as a Lead In qualtiy will you join that company? I mean to ask what do you look while making a change Designation, Money, type of work? what?

IT Engineer: right now I looking for more money because I took a home loan and want to repay that fast and I know that kind of profile I am looking for ... is always challenging
Madhukar: How will joining Company X help you in that? What is your total expenses at present location after spending on room rent, food, transport and all like going to your hometown once a month etc......If at Company X you have a higher take home salary then on the other hand you might also have higher expenses per month in the new city as well.

IT Engineer: thats what making me rethink each and everytime because my major concern is this only.
Madhukar: means i am interested to know how does joining Company X gaurantees you that you can repay home loan quickly.

IT Engineer: yes true
Madhukar: Hmm which means designation does not matters?

IT Engineer: it wont help me, Designation do matter to me.
Madhukar: How? Don't you feel that all these designations are the fancy names given to just seperate out people and make the people happy....Ok what are your responsibilities as a senior QA engineer at present role..and how do you gaurantee that being Senior QA at Company X also will give you same? Might be Company X's definiton of Senior QA and the roles and responsibilities is quite different than other companies.

Madhukar: In India if you are not promoted to a higher level by attaching a fancy tag to your
Visiting Card then people start to change the companies and thats the game companies play to lower down the attrition rate,....The Thumb rule in India is 0-2 Years Entry Level, 2-4 Years Senior Engineer, 4-6 Years Lead..., 6+ years Manager
IT Engineer: yeah you are absolutely correct as i told you that I was not at all expecting the call from Company X once i received a call I thought why not to give an interview and see where i stand in the market clearing the first round gave a confidence and then clearing a second round gave me more confidence

Madhukar: Ok, i was just asking the questions because of my curiosity.
IT Engineer: ok

Madhukar: It was not meant to demoralise you or stop you from taking any decision....I just wanted to know if you have thought from all angles and was just making you realize the impact which this decision can have at a later stage.

The discussion continued for a long time and i made few observations: Software Engineers in early career just dont realise the big picture and want to change companies because of money, better work, onsite and several other reasons. We should always see as to what extent we fit in the next company, what goals and benifits are fulfiled and finally we should not move by the feeling that better things exist on the other side. Better things do definitely exist however they needs to be explored properly.

Friday, January 11, 2008

The uTest Testing Community: Get Paid for finding Bugs

Found out a pretty interesting new concept uTest, check it out.

By joining uTestTM you will become a member of the world's first online community of software testers and gain access to the uTestTM Testing Platform. As a Tester in our community, you will be the first to try exciting and innovative applications, many of which are pre-released and have yet to be introduced to the public. Our network of testers (whose experience ranges from common consumers to highly trained testing professionals), forms a community from around the globe dedicated to experiencing and helping perfect bleeding-edge technology.

Our Testers enjoy extremely flexible hours - so you can work when you want to and at your own pace. In addition, the uTest Testing Platform can operate on any machine connected to the internet, which means Testers can work anywhere they want: from the comfort of their own home to their local coffee shop.



Thursday, January 10, 2008

Appreciation and its impact......

Was thinking from a long time to write some thing and after a long time here i am with my thoughts on a topic rarely in the discussions. The inspiration for writing this came from Shrini whose views i love to read.
I had a recent conversation with Shrini on testing ERP applications which made me write my experiences.

ERP , the vast ocean of Enterprise applications, I was doing this testing for an ERP application some time back and due to a showstopper in the application for which i was developing automation scripts i thought of doing some testing of my own. To my surprise i was finding bugs which could have been caught long time back and which were so obvious, this raised my curiousity to test it more and i started doing exploratory testing. As i was finding more and more bugs my excitement was rising.
I managed to find some 50 odd bugs in 2 days and was extremely happy that i found some real trivial and amazing bugs in the application. Then started the real challenge of explaining the management about the importance of rapid testing versus scripted testing as somehow i have the feeling that scriptd testing is considered by the management as a robot which will perform all jobs by itself and once completed we can run them to find if any bugs have been injected when a new build was released.

Here goes the conversation:

Me: I would propose ET (Exploratory Testing) for testing this application as i dont find any value addition in doing scripted testing which i have been assigned which does not finds the bugs which ET can find.
Manager: Why?

Me: Its been more than 2 years since the development for this application started and and still their are so many bugs in the application which should have been caught by now. I would like to know from you what are the benifits of we doing scripted testing when it is serving no purpose.

Manager: Well i feel that their is already another group who is looking after this aspect of the application.

Me: You mean to say after the exploratory approach of testing.
Manager: Well i don't know what is exploratory testing ( The manager is from a development background), i only can suggest you at this moment that we have some goals to achieve and we have clear directions from higher management of how we achieve the same hence your priority should be to perform scripted testing.

Me: But it is not finding any bugs which should be our goal.
Manager: I already told you that other groups are taking care of this.

Me: ( I knew no other such group is doing this kind of testing) I would like to know who is that other group and if such a group exists then i would like to see how they are performing testing as according to me these bugs should have been identifed by now.
Manager: At this point i have no clue as their are so many teams working acroos the Globe so it would be difficult to identify and provide you the details, hence you pay attention to your goals for this year that is scripted tests.

Me: Are you aware of something called “Pesticide Paradox”?
Manager: No, what is this pesticide killer you are taling about.

Me: Well bugs are like pesticides in software terminology. As pesticides over a period of repeated application of a specific pesticide acquire immunity to it and fail to show their presence, in the same manner software bugs fail to show up and become immune to repeated application of specific Test cases which we call scripted testing. Might be some bug is lying in the system which our tests dont navigate through at all.

Their are still so many bugs lying in the application but since the way we are looking at quality is just to make sure that the important business processes keep on running hence we are not seeing the so many hidden bugs which makes me more concerned about the quality of the product.

The discussion continued for a long time and the output was negative as due to vastness of ERP applications generally quality is given a backseat and thing which is made sure is that the main functionality or the Business flow should work properly which the customers are going to use.

Customers also being not taught about the application and with limited knowledge just know what they have to follow to complete a scenario and follow the same steps. The scripted tests also are made sure so that they cover the business flow. So many bugs are left in the application and no one complains about it simply because the business scenarios are running and thats what customers are going to use, but what if the customers deviate from the normal process and do a bit different?


The attitude for quality should be present during testing and this should be well supported at an organizational level, for this we still need to educate ourselves a lot and give due respect to quality.

The point i like to highlight here is:

If the software testing team gets appreciation and their views are taken with due respect then we have many passionate testers who can do wonders for the company.

Sadly we dont find such managers and stakeholders, everyone is busy in thinking that why should i put my head in this dirty work, let me do what i am assigned to and achieve the objectives and goals assigned.

Appreciation and freedom given to express views and taking prompt actions on the same goes a long way in building good testers.

What do you say?

Madhukar.

Saturday, June 2, 2007

Testing Google- Boundary Values Explored

Another interesting feature which i tested was Boundary value of Google's search box. I was reading this interesting post by Debasis and i found out the work done by him really impressive and thought of doing my own boundary value exploration for Google's search box...


Debasis has mentioned in his analysis that he found out the maximum no of characters which can be pasted in the box is 2048 and on further analysis he found that it is 1980 characters which when passed will result directly on a "Google 404 Bad request Page".


Here is what i have found:


I found that it is not the exact no 1980 which can cause the "Google 404 Bad request" page. I found it to be something else. I found that it is not the no. of characters which really matter for getting the bad request page, however it is the positioning and placements of few special characters like #,@, ( to name a few which were causing bad request page.

We need to definitely explore a bit in deep to find out what exactly causes bad request page because after my initial investigations i am now sure that it is something which needs time and deep investigation to be done to find out the exact reason.


I found out that i was able to reproduce the "Google 404 Bad request" page several times using different combinations and different placements of characters in the search string. The interesting part was that whenever i included a ( after several @ symbols in my search string, the result would automatically shift the line starting with ( on the next line. For more clarity check out the snapshot here.


I was surprised that why the search was displaying (, # and few other characters in next line of search results. I found out that increasing the no. of #'s in the search string was increasing the chances of minimising the 1980 barrier required to get a Bad request page. I pasted the string mentioned below which has 770 characters in it..


_________####_______________#############____________________+++++++++++++++++++++++++++++++++++@@@@@@@@######@@@+++++@@@@@@@@@@@@@@@@@(((((((((((((((((((++++++++))))))))))((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((@@@@@@@@@@@@@@@@@((((((((((((((((((((((((@@@@@@@@@@@@@@@@@@@@@@@))))))))))))))))))))))##################################################################********************************(((((((((((((((((((((((((((((((^^^^^^^^^^^^^^^^^^^^^^^((((((((((((((((((((((((((((((((((((((((=====================================================================================================================================((((((((((((((((((((()))))))))))))))))))))))))))))))))))))))))))))))0#################################################(######


Paste the above string exactly in the Google search box and you will find that when we press Search Button we get the bad request error. Now if i reduce the above string to 769 characters i was not getting bad request error.

The interesting part was that sometimes this bad request error comes also when i have 765 characters and sometimes it does not comes.


Strange but true.. I see definitely here a relation between how we place characters and which characters are at what position has definitely to do something with bad request page.


I am sure that we can even further reduce the no of characters to get bad request page.


Output:


We need to explore more on what exactly causes Bad Request Page..


Happy Googling..
Madhukar.











Thursday, May 31, 2007

Immediate boss is the reason people stay and thrive in an organization. And he ’s the reason why people leave.

Sometimes back i was going through this amazing article written by Azim Premji which truly reflects state of the IT Companies in today's world....I am amazed to see the trueness which this article reflects...You might probably feel that you also have encountered this situation sometimes in your life...


What do you say...

Azim Premji, Wipro
Every company faces the problem of people leaving the company for better pay or profile. Early this year, Arun, a senior software designer, got an offer from a prestigious international firm to work in its India operations developing specialized software. He was thrilled by the offer. He had heard a lot about the CEO. The salary was great. The company had all the right systems in place employee-friendly human resources (HR) policies, a spanking new office,and the very best technology,even a canteen that served superb food. Twice Arun was sent abroad for training. "My learning curve is the sharpest it's ever been," he said soon after he joined.
Last week, less than eight months after he joined, Arun walked out of the job. Why did this talented employee leave ? Arun quit for the same reason that drives many good people away.
The answer lies in one of the largest studies undertaken by the Gallup Organization. The study surveyed over a million employees and 80,000 managers and was published in a book called "First Break All The Rules". It came up with this surprising finding:
If you're losing good people, look to their immediate boss. Immediate boss is the reason people stay and thrive in an organization. And he's the reason why people leave. When people leave they take knowledge, experience and contacts with them, straight to the competition. "People leave managers not companies," write the authors Marcus Buckingham and Curt Coffman. Mostly manager drives people away?
HR experts say that of all the abuses, employees find humiliation the most intolerable. The first time, an employee may not leave, but a thought has been planted. The second time, that thought gets strengthened. The third time, he looks for another job.


When people cannot retort openly in anger, they do so by passive aggression. By digging their heels in and slowing down. By doing only what they are told to do and no more. By omitting to give the boss crucial information. Dev says: "If you work for a jerk, you basically want to get him into trouble. You don't have your heart and soul in the job."

Different managers can stress out employees in different ways - by being too controlling, too suspicious, too pushy, too critical, but they forget that workers are not fixed assets they are free agents. When this goes on too long, an employee will quit - often over a trivial issue.

And finally the Golden Words...
Talented men leave. Dead wood doesn't...

Wednesday, May 30, 2007

Add Value to your Automation....

Simplify Automation Creation and Maintenance....

I am sure anyone involved in automation testing would have encountered the situation below:

Your team is working hard on an application, lots of automation scripts are created, the test suite is running all fine when suddenly one day you wake up to find that all scripts have failed..
Nothing changed in your code..right? Then what caused this catastrophic failure..
Everyone is behind you asking for the reasons why it failed and you yourself do not know what happened...Strange isn't it...? How can a person who has written the scripts be unaware about why they failed in the night..
I will tell you why....

1. The scripts were procedural test scripts which were just recorded and replayed with some checks in place, some loops, some if else statements and so on...

2. These kind of scripts are difficult to maintain when the application changes, and in development the application changes quite frequently from build to build. Some new objects get added to the application, some old objects are not present, some objects have their properties changed and so on....

3. Testers generally have an attitude that automation tools are just about record and replay and nothing more, they are unaware about the capabilities which they can bring into it.

How do you overcome all these issues then?

You can simplify maintenance by avoiding to fall in the pitfalls of poor record and replay scripting..which will not lead you to automation success.

Automation becomes of little value when it does not provides useful information which can be reviewed by testers and most importantly my managers (after all managers are interested in the number game, in the % age of bugs covered by automation etc)...

Never rely on the tools built in reporting features. Create a library of functions which can be then reused by each test. Using the library the output can be given in so many formats. Find a few examples here.

1. HTML Files: People love coloured HTML pages, plus on top of it if you can add a little bit of Java Script to it then it can be more presentable.
2. Excel Sheets: Excel sheets can be processed easily by machines and people can do lots of manipulations with the data just by using few mouse clicks.
3. Database: Results can also be posted directly to a database and which can then be used to comapre with a pre existing standard template.
4. XML File: Another very popular medium of output.

Reported results should contain enough information which can answer the following:

What happened to the application when error occured.
Exactly at which point did the error occured.
Which script and which line of code caused the error.
What action does the Framework took when error occured.
How many errors were application specific and how many errors were due to errors in scripts itself.
How many test cases ran totally?
How many passed and how many failed?

One more feature can be built into the framework..Snapshot of the application can be taken and stored whenever an error occurs so that it becomes more easy to debug the errors.
Snapshots also provide a real proof of where the error occured. Snapshots can also be customised with the Log File ouput, for example the time stamp in the log file for a particulat step can be appended to the Snapshot file name so that it becomes easier to find the actual state of the application at the point when error occured.

Most Importantly remember:

An automation effort which runs for several hours is of little importance if it takes another several hours to figure out what went wrong and to interpret the results..

And yes i will be posting many new ideas for automation..so keep reading...

Happy automation..
Madhukar..