Compatibility Testing

Testing the functionality of an application in different software and hardware environment is called Compatibility testing.

Consider an example below,

AutoCAD with Win2000 was tested on Win2000 platform. Once it is tested, the product is sold but the customer who buys it uses in different platforms like XP, NT, Win2000, and Win98 etc.

Customer therefore (or) End-users therefore blame the s/w, they don’t think about the OS they are using because the application was tested on Win2000 and the user may be using it in XP.

Thus in order to overcome the above scenario, we do test the application in all platforms and then sell the product.

In the above example, we run the application in different platforms and if we find a bug in any of the platforms we have tested (say, Windows 98), then the s/w is given back to the developer for fixing the bug.

Once, the bugs are fixed, and then we have to continue testing not only on the platform which has bug, but also once again for all the platforms because fine tuning may cause a change in other platforms. Thus, maybe we may use one of the following scenarios after fixing the bug,

a) if we are sure of the area of the bug, then test only that area.

b) if we are not sure of the bug – then go for testing all the platforms.

Now, the question arises do compatibility testing is done for all platforms? – NO. Observe how it is tested,

Suppose only 8 TEs are present for testing, in this case –we don’t have sufficient people for testing on all platforms. So do we stop testing? – NO. Then how will we test – see below,

Now the TE will see for maximum use of platform by customers and do test for only those platforms.

In this example, TEs they look for most widely used platform i.e., in this case

XP, Win 2000, NT, etc. and do testing for all the above said platforms and

Do not test for the minimum used platform. If we are not testing other platforms doesn’t mean it will not work.

It will work on other platforms also, but if any bugs or issues arise – we are not fixing and not responsible for it.

For this scenario to be avoided, sometimes in the product they mention for which OS it supports. Now,

we can think where we can do Compatibility testing? – When we   cannot force the users to use an application, then we go for CT. 

Now, let us consider an example below to see where and how we do CT. (Figure in the next page)

If we take HP, IBM, SUN – these are h/w companies – they manufacture servers, when they want to sell the product – they can’t sell the product only – therefore these companies have their own servers.

SOLARIS and UNIX can be tested by using Open Source Internet Explorer (IE)

By default, IE comes with OS. IE is always tightly embedded with OS, so we cannot remove OS from IE [if we try remove, may be the application may not work].

IE generates lots of revenue to the company as it is robust and has got loops of security in it.

Opera claims it is the fastest browser. IE is not Microsoft original product [it is actually got and combined with other applications]

Developer builds www.shaadi.com and installs it in the testing server.

For the above application, (shaadi.com) – 4 Test Engineers are given 5days for testing. The testing is as follows by 1 engineer which is the same for the rest of the testing team,

1 TE opens OS – WIN 98 – Test for IE 5.0

Opens OS – Win 98 – Test for IE 5.5

Opens OS – WIN 98 – Test for IE 6.0

Thus for 4 TE to test 1 OS and 8browsers within 5days, it takes 40days to test in 1 OS.

So far all OS to test 40*6 = 240, it takes 240days for the TE to test the application.

Like this list of all flavors of platform are available, but we have to test for the OS as browser which is in maximum use. Thus for Windows, we do test for WIN 2000 and Windows XP which is widely used

                

Similarly Mackintosh is used by Apple computers and they sell in Europe and US.

Similarly for browsers we do test for the following cases which are widely used.

IE – for 5.5 and 6.0 – we do CT

NN – for 6.0 and 7.0 – we do CT and also for Mozilla Firefox we do CT

So, we always think about ROI (Return on Investment) and do the testing and also follow the criteria below,

·         Number of users are less – we don’t prefer CT

·         Number of users are more – we prefer CT

WINDOWS 2000 Professional – normally corporate companies

Now, let us see the set-up of the environment for AutoCAD testing with OS

Install Win 2000 in your machine and start testing for CT (1 of the platforms)

Similarly test for the next platform – IE (IE 5.5 and 6.0). Open the IE 5.0 and test for it.

In this window, allow only 1instance of IE i.e., 1^st we should test for IE 5.5, remove and then allow many instances (i.e., to have both 5.5 and 6.0 at the same time)

After we finish testing for the 2 available most widely used platforms, we move on to the next platform NN 6.0 and 7.0 – open NN 6.0. Here, the window allows 2instances i.e., to have both 6.0 and 7.0 at the same time.

How to handle bugs if we are testing for different platforms

Testing functionality, integration and end-to-end testing on various platforms is what we do in Compatibility testing.

Compatibility issue – It is a feature not working in 1 OS, but working fine in all other OS. It happens because program written by developer is not working in only 1platform, but working on all other platforms.

The compatibility issue is sent to developer for fixing the bugs and sent for testing. Here, to retest the application, we have to uninstall the OS and not the browser and re-install the OS and test.

Functionality issue – also called Functionality defect issue – a feature not working in all platforms/OS.

It is also possible to have different platforms in the same machine. For this, each platform(OS) should be in different drive like below. We can run 3OS in same m/c in different drives – C: Windows 98, D: XP,

E : Windows 2000

Hardware Compatibility Test

·         Test on different processors,

·         Test on different RAM

·         Test on different Motherboard

·         Test on different VGA cards

In Test on different processors – we test for make – Intel, AMD processor and also test for speed – 3.1GHz, 2.7GHz

In Test on different RAM –we test for make – Samsung, Transient and also test for size – 1GB, 2GB

In Test on different Motherboard – we test for make

In Test on different VGA cards – we test for make – Nvidia, ATI