Welcome to MSDN Blogs Sign in | Join | Help

Test Automation with Ruby: Don't drink the kool-aid

I was stuck at home this weekend with house painters pondering how best to spend the day sitting at home, so I decided to actually play around with the Ruby scripting language a bit. I will admit my bias in my previous post about the Ruby scripting language was based on business justifications and personal preferences towards the C family of languages. But, because Ruby is still being pimped by some experts in the industry for test automation, I figured I should understand more about the mechanics of language other than it's lack luster appeal by many professionals in the software industry.

 

I didn't delve very deeply into Ruby. A few chapters of Hal Fulton's book The Ruby Way and some additional research on the web confirmed my suspicion that Ruby for all purpose software test automation is simply not a good investment in your time given the other alternatives.

 

International Support Sucks!

 

If you are stuck working on small projects in English only then this probably won't bother you too much. But, if you realize the majority of the world does not communicate in English then Ruby's lack of international support simply sucks. Unless there is some hidden secret in the Ruby society the fact is Ruby can't do Unicode. Yes, you can store UTF-8 encoded data in 8-bit encoded strings, but many Ruby String methods assume single byte encoding. I have to admit this completely blows my mind. UTF-8 is a popular encoding form of Unicode; however, it is only one of several Unicode transformation formats. So, forget about UTF-16, and definitely forget about testing Unicode surrogate pairs or conformance to the GB18030–2000 Chinese encoding standard. So, what does this mean for testers? String testing is basically limited to a very small subset of characters.

 

Multiple ways to initialize arrays (and just about anything else)

 

Hal Fulton lists the following 3 ways to initialize the array class method.

            a = Array.[](1,2,3,4)

            b = Array[1,2,3,4]

            c = [1,2,3,4]

I don't know why this would be a 'cool' thing to do, but I do know that it increases confusion, and potentially adds to the cost overhead of white box testing and code maintenance.

 

I guess it's pretty cool that arrays in Ruby contain objects rather than data types, and an array in Ruby can contain integers, strings, etc. I am not quite sure why this is a positive because I can't think of a situation where I would want to iterate through an array with disparate data types.

 

Cryptic codes

 

If Hungarian notation wasn't bad enough, just stir in some Ruby identifiers into the cryptic pool of sludge. Let's see the # sign is a comment if it is the first character in a new line or after a statement (which of course doesn't end with a semi-colon or other signifier), otherwise it is an embedded variable. And what is the deal with variable scope declarations? The @ sign is used to declare instance variables, using @@ signs declares a class variable, and of course the $ sign declares a variable that is global in scope. Hmmm...I am thinking the access modifiers 'public' and 'private' in C# make a lot more sense, but of course I have to type all those letters instead of just pressing the shift key followed by one or two characters.

 

Useless keywords

 

For all you Ruby experts, can someone please explain the purpose of the unless keyword? It seems to me to be another bit of useless confusion. Let's see if I have this right in Ruby:

 

            if x < 5 then

                        statement

           end

and

            unless x < 5 then

                        statement

            end

 

are identical! What's the point?

 

Syntax?

 

And what's the deal with 'downcase' and 'upcase' instead of 'lowercase' or 'uppercase?' Is downcase really intuitive to anyone other than Ruby zealots?

 

I could probably go on, but frankly I am convinced that Ruby is not the way.

 

So, that's a day in my life that I will never get back. But, all was not lost because in-between chapters of the Ruby book I got to watch the paint dry on my house.

Published Friday, August 25, 2006 5:01 AM by I.M.Testy
Filed under:

Comments

# re: Test Automation with Ruby: Don't drink the kool-aid

Surely "unless" is translated as "If Not"? In which case, "if x<5" is NOT the same as "unless x<5". i.e. "unless x<5" means "when x is NOT less than 5"?

I agree that the wording is strange, but surely this is just an alternative single-word approach? All languages differ, it's just a case of getting used to a different set of words?

Regarding the "downcase" and "upcase" - Excel has had "UCase", and "Upper", so I guess "downcase" could be regarded as one logical opposite. Other languages have "UPCASE" - Delphi and php?

(PS - I'm a Test Manager, not a coder, and have no knowledge of Ruby)
Tuesday, August 29, 2006 12:10 PM by Elijah

# re: Test Automation with Ruby: Don't drink the kool-aid

Hi Elijah,

WRT to the unless keyword in Ruby, looking at a list of keywords I would also assume different behaviors. But, that is simply not the case.

I agree that each language uses different words. (I also never really got the DIM thing in VB for example.)

My key point is that having alternative keywords that do the same thing simply obfuscates the code.

I just thought the upcase/downcase thing a bit strange. The real deal breaker for me is the way Ruby deals with strings. Much of the data we deal with today is Unicode and there are multiple transformation formats of Unicode(of which Ruby partially supports only UTF-8).

Ruby may have some benefit in the web development community for limited applications. But, as a useful language for testers, Ruby simply can't do what I need (especially since the web is designed to support the Unicode.)
Tuesday, August 29, 2006 9:48 PM by I.M.Testy

# re: Test Automation with Ruby: Don't drink the kool-aid

Totally agree with Elijah in what was said regarding the 'unless' and UCase scenarios if it makes it easier, view unless as IF NOT. In addition, if I understand you correctly, you cannot see the benefit of the mixed array types "an array in Ruby can contain integers, strings, etc. I am not quite sure why this is a positive because I can't think of a situation where I would want to iterate through an array with disparate data types."
Surely a form entry where you wish to be able to collect mixed data types (ABC,123...etc) makes this function invaluable. I don't see how you cannot see the benefit of this?!?

Again, I know nothing of ruby so I may be speaking out of turn, it's just that most of the things you mention seem either standard in coding languages across the board, or a good idea.
Wednesday, August 30, 2006 5:23 AM by Lewis Cameron

# re: Test Automation with Ruby: Don't drink the kool-aid

Hi Lewis and Elijah,

You guys are right and I stand corrected. Thanks for driving that point. Sometimes I can be a bit thick-headed. (I'll blame it on the latex paint fumes :-).) Hal Fulton's example which is specifically:

if x < 5 then
 statement1
else
 statement2
end

unless x < 5 then
 statement2
else
 statement1
end

Basically, this is the same as doing

if (condition)
   statement 1;
else
   statement 2;


if (!condition)
   statement 2;
else
   statement 1;

I know...that darned little exclaimation point sometimes gets overlooked and is not understood unless you know the cryptic codes used consistently among languages such as C/C++, Java, C#, etc.

WRT to the arrays of disparate data types I understand your example to collect information from a form, but then what efficient task are you going to do with it? You can't sort it, you can't loop through it efficiently, etc. So, I guess I should have clarified and stated that from a test automation perspective I don't see the benefit.

But, the things I point out are not so standard in coding languages across the board with the exception of perhaps the cryptic codes.

Java, C#, C/C++ all support Unicode and the various transformation formats for encoding strings of Unicode characters. Most languages do not offer multiple ways to delcare an array object (or other declarations), and the only languages that have the "unless" keyword are Ruby, LISP, and BOO.

Again from a test automation standpoint I stand by my statement that Ruby is not a good choice. WATIR may provide some great support for web testing, but what about testing Windows applications? I guess I will have to get busy on the benefits of C# post so folks can make their own decisions.
Wednesday, August 30, 2006 9:49 PM by I.M.Testy

# re: Test Automation with Ruby: Don't drink the kool-aid

1) Another motivation for "unless", I believe, is to permit statements like this:

do_this_thing unless x >= 5

which could be more readable than some alternatives--especially when the condition is based on a method that returns a boolean:

do_this_thing unless x.is_a_big_number

2) I don't know the Fulton book.  Amongst the Ruby community, it doesn't seem to have the same level of endorsement as "Programming Ruby", by Thomas and Hunt.

3) With respect to the complaints about array initialization, there are even more ways than the Fulton book lists.  The philosophy of Ruby is like that of Perl:  There's More Than One Way To Do It, linked to The Principle of Least Surprise.  Express something in a way that makes sense to you, and it's likely to work.    It's kinda like // and /* */ in certain other languages, or

x = x + 1;
x++;
++x;
x += 1;

You view this diversity of expression as a bad thing; some people like it.  It's a big world.

4) With respect to syntax, perhaps the author intended to emphasize that there is an action being performed on the string -- "downcase this, sucka!"  I don't know.  But if you don't like it, create a file called bj.rb, and fill it with a whole bunch of stuff like this:

class String
 def lowercase
    self.downcase
 end
end

and then

require bj.rb

in your program to make the language look just the way you like it.  (You can put it into your irb.rc, so that the Ruby interpreter can include your extensions too.  The interpreter, irb, is one of the wonderful features of Ruby.)

5) Whether a statement ends with a semi-colon or a newline is surely a matter of personal preference.  VB doesn't require statement terminators; neither do lots of other languages.  C and its descendants do.  Perl does.  Big deal. Rock on.

I've found using Ruby to be practical, pragmatic, and fun.  It's very handy in creating domain-specific languages, and WATIR is immensely powerful.  But Ruby is only a set of vise-grips; it's not a ratchet wrench.  Choose a tool that's appropriate to the task at hand.

You are incorrect when you assert that learning or using Ruby is not a good investment of my time, given the other alternatives.  How would you know?  You've never met me.  What you mean, I think, is that Ruby is not a good investment of your time, which I couldn't deny.

---Michael B.
Thursday, August 31, 2006 4:42 AM by Michael Bolton

# re: Test Automation with Ruby: Don't drink the kool-aid

Your points are well stated Michael, and I agree that all languages have their quirks, idiosyncracies, and some stuff that just makes us say "huh?".

And I haven't looked at WATIR. And testers who limit their testing to web interfaces may find WATIR sufficient to fit their needs.

It is interesting that no one has yet to address the issue of the lack of Unicode character support other than the limited UTF-8 support. The Unicode Standard supports 3 encodings, and there are several more defined. I guess this is ok as long as you only deal with one language and that language is English.

I also stand by my assertion as long as it includes the entire assertion which states "for all purpose software test automation Ruby is simply not a good investment of your time given the other alternatives." By 'all purpose test automation' I am implying testing legacy Win32 applications, managed code applications, and weblications and services.

Learning alternative languages from a dev perspective has some merit and sometimes makes new ideas click. But, for non-coding testers considering learning a language for test automation to improve their marketability and/or future business needs I simply wouldn't suggest Ruby. (Insert blatent plug for C# here.)

(Mental note to self: Avoid the use of the 'you' pronoun in writing.)
Friday, September 01, 2006 12:02 AM by I.M.Testy

# re: Test Automation with Ruby: Don't drink the kool-aid

I found the assertion that "There's More Than One Way To Do It" (as for the array initializations) "potentially adds to the cost overhead of white box testing and code maintenance. ", very interesting.

My thinking is about what is special about the internals of Ruby/Perl interpreters, which make them be able to "freely" support multiple ways?

Tuesday, October 31, 2006 8:57 AM by Girisan

# re: Test Automation with Ruby: Don't drink the kool-aid

Hi Girisan,

For many folks learning a programming language can be quite daunting, and the value of expressive languages such as Ruby can lower the initial hurdle.

However, expressive languages mostly benefit the developer writing the code because the language doesn't require a lot of constraints.

From a (white box) testing and maintenance point of view where individuals other than the developer are reviewing the code, the free use of expressive languages (without guidelines or standards) increases the cost of testing and maintenance.

We review our own code very efficiently (we wrote it). And, when we review someone elses code that follows similar coding guidelines it takes slightly more time. But, when we review someone elses code that is unfamiliar to us the time required is considerably greater.

This is one reason why many large companies (Microsoft, Phillips Medical, GE, DoD, etc.) who produce software with a long shelf-life and whose code will often be maintained for several years by developers other than those that wrote the initial code rely on coding standards and guidelines; coding guidelines help reduce the long term costs.

For small projects with limited lifespan, where the code is informally reviewed by peers, and white box testing is limited or non-existant the cost of such overhead may not be prudent. Expressive languages certainly provide rapid production in these situations.

Tuesday, October 31, 2006 2:17 PM by I.M.Testy

# I M Testy Test Automation with Ruby Don t drink the kool aid | Wood TV Stand

PingBack from http://woodtvstand.info/story.php?id=11531

Sunday, May 31, 2009 5:54 PM by I M Testy Test Automation with Ruby Don t drink the kool aid | Wood TV Stand

# I M Testy Test Automation with Ruby Don t drink the kool aid | Wood TV Stand

PingBack from http://woodtvstand.info/story.php?id=52719

Tuesday, June 02, 2009 8:39 PM by I M Testy Test Automation with Ruby Don t drink the kool aid | Wood TV Stand
Anonymous comments are disabled
 
Page view tracker