First, Smart Identification (SI) is an algorithm within QuickTest that kicks in when, within the code and repository, two elements look the same to QTP. SI then takes a peek at the the object in question, in an attempt to sniff out a property to distinguish an element from its look-alike cousin. If it finds such a property, it will let QTP know what it should be doing with that element. In a nutshell, it’s doing what the tester should have done in the first place: Looking for a good, unique property of an element, so that QTP is able to take a look at the script and immediately know which object on the page it’s supposed to be carrying out the task at hand on.
So getting back to the question at hand, the answer is ‘Yes’, because it can be useful, particularly to someone not particularly familiar with the ins and outs of QTP, ie. a novice, who may not know to how or when to employ the use of descriptive properties. It can also help out as a helpful reminder in case of an oversight.
And the answer is ‘No’, because the experienced tester should have a feel for when and where to properly describe all the elements used in their test. It’s also a bad idea in case a tester does not know how to read the results correctly, and assumes because their script is ‘working’, they did everything correctly.
In the end, the usefulness of Smart Identification probably depends on the experience level of the tester more than anything, rather than being a black and white issue.
While working on my current project I have been writing programs that test the functionality of our clients upgrades they are making on their main client database. Now, keep in mind that when companies re-write their applications and databases…they have two environments. They have their LIVE and they have their TEST. I use the TEST environment very frequently but, when I write a program that does not save information I test it in both. This week I ran into an issue, one that completely blew me away. It seems that its always the little things that aggravate you the most.
Once I was done writing my code I tested my program in our clients test environment which is accessed through Internet Explorer. Worked like a charm, on the first try even! In the programming world, that does not happen very often. Anxiously, I switched over to their live environment which is also accessed in Internet Explorer. I was not so lucky this time around. This code just worked, flawlessly, but now something is broken. By broken I mean every text field, check box, and button I was verifying would not be recognized by QTP.
I quickly go back to the test environment and run it again. Unbelievably, it fails in this environment as well. Right away I knew that it had something to do with switching the applications. The issue that perplexes me the most is that when I switch between test and live environments, all I change is ONE letter in my URL.
With this in mind, when using QuickTest Professional and you are switching between environments that use the same interface (like Internet Explorer), always make sure to close IE completely and do not simply just change the URLs. It will save you time, energy, and your sanity.
J9 Test Asset Developer
On my current project, I have been updating business components that were recorded against an Oracle platform with a JAVA presentation layer. Initially, due to some object recognition issues with the Oracle add-in we were forced to use just the JAVA add-in by itself.
Since these component recordings the client has implemented a patch to their Oracle instance, that has “corrected” a lot of the object recognition issues that we were having. Now we are able to use the Oracle add-in thus enhancing QuickTest’s functionality against the Oracle forms.
We originally encountered many aggravating issues with using the JAVA add-in only, for example all edit field entries were being recorded with the same object name, and all were using the SetSecure method opposed to the Set method used to enter data.
Once the Oracle patch was implemented, one problem I encountered was that I was unable to record an action that required clicking on a sub-tab called “Collections.” This tab was to be activated in order to interact with the fields it contained. Normally QuickTest has no issues at all with selecting items within a tab strip (in other environments), but on the Oracle environment it simply would not register. I “spied” on the object and found no identifying characteristics except for the name. With such a powerful identifying trait why would QTP not recognize this action when recording? My first inclination was to do Low Level Recording. However, this is only used as a last resort so I decided to try adding the tab to the repository and added …OracleTabbedRegion(“Collections”).Select to my code.
Ultimately, this was the correct fix. It is important to remember or search through the vast amounts of objects and methods when working with advanced scripting. A little extra effort can truly enhance the quality of your work.
J9 Test Asset Developer
I heard the phone clang down and my colleague Steve distraughtly mumble “She’s going to kill the fish.” His wife called to tell him about a phosphorus problem in their fish tank at home. She’s a medical researcher, a biologist by training. Steve’s first reaction when she told him there was a phosphorus problem was to ask if she had in fact done a phosphorus test. No, she said, but she’d run through all of the other chemical and algae tests, so of course it had to be the phosphorus and thus she’d started adding more phosphorus to the tank — they’d know in a few days if that was the problem. Steve, imagining coming home to a tank of dead fish, was not impressed that his scientist wife had failed to use the scientific method at home.
It’s so often like that in technology as well. Despite years of rigorous training to use the scientific method to guide our actions (it is called “computer science” for a reason), it’s easy to throw all that away when faced with a challenge. A customer came to me the other day asking about monitoring tools to help with a production triage situation for a failing web service. A developer assigned to the task interrupted us saying that a fix had been deployed ten minutes prior and it looked like it was working. Let’s reflect upon that:
a) No load or performance testing scripts existed for this web service.
b) No monitoring or profiling tools had been deployed with this service in either a pre-production or production setting.
c) A hopeful fix had been hot-deployed to production and left to run for a mere ten minutes before victory was declared.
d) No permanent monitoring was put in place to prevent the next occurrence of the problem.
e) Apart from a few manual executions of the service and a face-value assessment by one individual, no further validation to correlate the fix with the perceived problem occurred.
Chances are good that Steve’s fish will be fine, but can the same be said for those cases where we play roulette with mission critical IT systems? Just as in the case of Steve’s fish, there is no legitimate reason for a lack of objective, quantitative analysis except basic human apathy. Anyone who has ever taken a statistics course or been face-to-face with a serious production issue knows that just because many other tests have ruled out many options does not mean its safe to jump ahead and make assumptions just because of gut feeling — why abandon a working method for one that brings doubt, risk, and exposure to criticism? Run the phosphorus test and let the results be your guide.
It is nothing new for us to be constantly developing new educational tools. Demos and lab materials for trainings on site, or content for our evolving KnowledgeBase that augments the HP software support we provide to our customers. But the videos are the biggest hits so far. They pack a three minute punch of information without leaning on those lazy powerpoint icons. Check ‘em out.
Business Transaction Management in palatable terms (no yawning required):
http://www.youtube.com/watch?v=49tQ9BpnrT0
In case you missed the first one, here it is:
Why J9? Well, since you asked…
http://www.youtube.com/watch?v=FjPlvO01SmA
Please rate them! We’d love to get some feedback on how well these videos connect with you and for god sakes, if they are still boring, please let us know.
That question was the lead in to a discussion I had with a colleague this week. He had been interviewing someone for a performance testing role and that was the key question that could make or break a candidate. The typical response goes something like “I’d start with one user, then move on to five, then ten, then 50 then 100, then… all the way up to 4000.” While the most common answer, this is entirely wrong. This kind of common yet broken testing process explains why the group of us that joined the conversation could each retell case studies of customers who had spent multiple years (and millions of dollars) on failed testing efforts.
The right answer goes like this:
a) Ask the hard questions
How many of the 4000 users are concurrent users and what is their use pattern? For example, many batch billing systems do nothing for 29 days per month, but then run through a massive number of transactions on the last day. Other systems have limited daily use until 5pm when their user community arrives home from work and then signs in. Are the users spread across multiple timezones?
If the data to discern the number of real concurrent users isn’t available, that actually means two things to our project:
1) A separate project is needed to put in place tools to capture user behavior. The lack of such information can cause poor decisions in the areas of testing, capacity planning, security, and product usability design and functionality.
2) If no such data exists and the 4000 number simply means we have 4000 users in our database, we can now back into a more realistic upward bound through some basic calculations.
b) Functional performance test
Start with one user as a means of functional performance test. This enables you to validate your test cases and test scripts and flush out any immediate functional problems with the application(s).
c) Longevity testing, peak testing, failover testing
There are a variety of other tests with greater pertinence and validity in understanding the application’s serviceability than simply running through the same script with a randomly increasing number of virtual users.
d) Load and Performance testing
If we’ve determined that simply starting with one user and continuing to double isn’t the right process for load testing our application, then what is the right heuristic for getting to the Nth user? The answer is that it doesn’t really matter, as we’ve determined, in effect, all of the above through the answers to our questions about the user community. If we have 4000 users in our database but don’t know how and when they use the application, a test of 200 users as a top number is just as valid as a test of 2000 users. Using these numbers though, one can arrive at some guidelines by looking at the length of a user day. For example, if our application is used by an internal business customer that only works standard business hours in the eastern time zone, then we can surmise a roughly 8 hour work day, 5 days per week. Take 4000 users, divided by 8 hours, we can take an educated guess that there are 500 users per hour. Take an 8 hour day, multiply by 60 to get 480 minutes, divide the 4000 users by 480 and we can surmise that at any one minute interval there are likely to be 8 users on the system. In the absence of further information about our user community, we now have real, actionable numbers to test against. Rather than the dozens and dozens of incremental tests we were potentially facing, we can now break our cases into one user, 10 users, 500 users, and anything above that is essentially to discover the upward bound of our capacity.
These steps are a productive tool to improve the quality of your testing, as well as a great way to gain new insight into the candidates you interview.
J9 Technologies, Inc. Announces Migration Solution for OpenView Internet Services Users
J9 Technologies, Inc. announces a limited-time migration solution for Hewlett Packard customers currently using OpenView Internet Services (OVIS). As of December 31, 2009, current OVIS customers must migrate from OVIS to HP’s BAC and SiteScope solutions in order to receive licenses, customer support, patches, and updates from HP. As a result of the Mercury acquisition, HP announced that it is dropping support for the OVIS solution in favor of the Business Availability Center (BAC) and SiteScope solutions. HP is offering, free-of-charge, a license exchange from OVIS to BAC and SiteScope.
J9 offers expert services to streamline the customer’s license acquisition process, pre-migration planning, end-to-end migration from OVIS to BAC, and identifying best practices for ongoing utilization of the new BAC solution. With J9, the migration process is not just one of conversion, but evolution. J9′s solution reduces a customer’s time to value risk during the migration process.
J9 works with each customer individually to determine their current state, identify gaps and provide a plan for migration. Throughout the process, J9′s experts partner with the customer to ensure the migration not only replaces their current state, with no gaps, but allows the customer to be positioned to expand their capabilities utilizing the enhanced BAC platform. J9′s services expand beyond OVIS migration solutions to offer a complete range of basic and advanced BAC implementation services and training programs designed to enable the customer to take full advantage of the power of the BAC platform.
Business Availability Center brings a robust platform to address a broad range of application environments. BAC’s rich feature-set offers customers an enhanced platform addressing their Business Service Management needs and initiatives, including application diagnostics, service level management, business transaction management, and superior dashboarding capabilities. The BAC platform is capable of supporting a wide range of application environments beyond standard web-based applications, resulting in enterprise-wide coverage for the customer’s most business critical systems, such as ERP/CRM, E-Mail, SOA, Web 2.0, and client-based systems.
About J9 Technologies
J9 Technologies is a certified Hewlett Packard Gold Partner, specializing in Business Service Management and Application Performance Lifecycle management.
Offering strategic consulting, training, installation and ongoing support for HP software products such as SOA Systinet, Service Test, Business Availability Center, Diagnostics, LoadRunner / Performance Center, Real User Monitoring (RUM), TransactionVision. With a focus on application diagnostics, composite application management, and business transaction management, J9 Technologies strives to ensure that customers can quickly identify the root-cause of issues and minimize mean time to resolution.
We are just catching our breath after few a long and fruitful days in Las Vegas, where HP held it’s annual Software Universe conference. We exhibited, we rolled out our OVIS to BAC migration offering, we chatted, we happy hour-ed, we presented, and after it all, we slept. For a long time.
And now here we are, in all of our glory, having emerged from the gambling flames of software sales and services just a little bit tougher, a little bit wiser, and a feeling lot more connected. Thanks to everyone who came by the booth (and picked up one of our lovely one-handed bottle openers pictured above) and who attended our happy hour event. It really was a blast getting to be face to face with all of these people we work with everyday. See you next year!
Yet another Monday after a red-eye flight, at another new customer in some city far from home. I’m ushered into a conference rooms with a team randomly checking their watches and Blackberries, each tense, nervous, and wishing they were somewhere other than this. A project manager begins the briefing, which everyone has heard a dozen times before: the project is past its scheduled delivery, application performance is terrible, the users are angry, the sponsors are talking about pulling funding, and the order for the past month has been forced 15 hour days and weekends. This afternoon they want to begin another series of load tests and they ask me if our team of consultants will be able to help them. My only question is: What are you testing?
It’s a simple question. I find myself asking it several times per day with customers and sadly the usual answer is a blank and distant stare. The reality is they don’t know, but it’s not their fault. To be honest, I’m not entirely clear myself on how the state of the industry reached this point. The trend though goes like this:
1) Testing happening just prior to deployment, often not even starting until the same week.
2) Tests being run by an operations team or perhaps by a lone consultant, without the involvement of a business owner who understands the functionality of the application.
3) A random mishmash of tools, some properly licensed, some not, downloaded and used without any training or guidance.
4) Test cases being made up on the fly – “Hey, what if we try this?” without any rhyme or reason as to the validity of the test.
5) Little or no written record of the test cases executed or their results, leading to duplicate efforts, errors, and a lack of objectivity.
It all really boils down to that same question — if you can’t answer, objectively, the reason for a particular test run, then your efforts are for nought. Once you are on that treadmill of test run after test run, it’s impossible to get off without a hard reset; the only objective test at that point will come when the application is introduced to the real world.
Once an organization has reached this point, where production firefighting is a daily routine, the only way to bring order back is to reverse the list above. Testing must be an integrated part of any development or implementation project. Developing a written, comprehensive, repeatable test plan requires input not only from technical teams but also from end users and business analysts who understand the use cases — both common and edge. Only then will you know what you are testing.
Shortly after I wrote the last entry, about tools that get purchased and not used, I realized that I hadn’t really described the reasons why this happens. Sure, there are reasons like insufficient training, inadequate tools for the problems at hand, or organizational ownership and politics. But the most common reason is far more human and harder to mitigate: apathy.
A friend of mine is a Six-Sigma and Toyota Process expert. He knows and understands quality and process control methods inside and out. I asked him about the barriers to organizational adoption of these practices. His answer was very telling. It wasn’t money, time, tools, training, or anything measurable. Instead his answer was very simple — because people don’t like being told what to do.
There’s a parallel here. Unless a tool, process, practice, or even idea fits within someone’s world view, they simply aren’t going to adopt it. It’s even worse if they feel like something is being forced upon them, rather than being an engaged participant. Even if they feel invested in the tools, good old fashioned apathy can take hold.
Why? Incentives and consequences. Its just tremendously easy to become apathetic if there are no incentives or consequences to our actions. Many times after J9 consultants find a problem with a customer’s application, we hear rumblings of “oh yeah, I wondered if that was a problem.” Easier to turn a blind eye then to invest ones self in learning a tool, exploring a problem, and coming up with a solution. Especially since in so many organizations that live by firefighting, volunteering is akin to accepting blame for the problem.
So, the guiding points here appear to be:
1) It’s imperative that all types of users are included in the selection and implementation process so that they feel invested.
2) Management has to continually communicate intentions and uphold the standard.
On this second point, here’s a really example of how this can be done in a way that emphasizes the point yet stays away from simply giving orders. Say for example it’s a Monday morning triage meeting and an outage that happened over the weekend is the topic. The questions isn’t simply “What happened?” but also “What did the tools tell you happened?” This is a means of continuous communication about the expected tools and the expected method, plus a means of continual assessment of the value proposition — if your tools are proving less than useful, then perhaps there is a legitimate reason why they aren’t being used.