August 2008
Sun Mon Tue Wed Thu Fri Sat
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31            

Sun, 02 Nov 2003

Rules Engine Problems

Rules engines at best are competent beginners in a limited, structured domain.

Discovering and programming the rules is a process requiring a very significant investment of time and effort, including both technical and business experts.

Rules engines can't handle exceptions that are not programmed in.

The current assertion that the improvements to rules engines are in "better interfaces" means only that we can get to the functions from across heterogeneous systems.

In most cases, experts don't in fact use the rules that might be discovered by interview and reflection.

Rules are always context free and cannot take into account the relevance of facts.

Ongoing maintenance of rules.

Deskilling

ceteris paribus

Expert human collaboration and judgment is required to correct and update rules engine errors.

References

Socrates, Euthyphro

permanent link

Build Machine Justification

The automated daily build and smoketest is one of the easiest and most cost-effective ways to increase software development productivity and quality. It is practiced by the most proficient software development shops.

High-level overview of the build process, from http://codefeed.com/tutorial/ant_intro.html

  1. Get the source. You may need to download or fetch the source from a source code repository. For this, you might need to know the tag or version of the source code you want to build.
  2. Prepare a build area. You will probably want to create a set of directories, perhaps according to some standardized directory layout.
  3. Configure the build. In this step, you will determine what optional components can be built based on the current environment. You might want to set build numbers and version numbers to be included in the build.
  4. Validate the source code. You may have a standard style guide and you wish to ensure all code conforms to this before you build a release.
  5. Compile the source code
  6. Build the compiled code into libraries potentially including non-code resources such as properties, images and sound files.
  7. Run the system's tests to validate the build.
  8. Build the documentation for the software. This may range from something as simple as collecting text files up to processing content through some form of publishing system to produce the documentation in its final form
  9. Package up all of the components of the software - code, resources, images, documentation, etc. - into a deployable package. You might need to produce several packages in different formats for different target users
  10. Deploy the software to some standard location for use or distribution

permanent link

Offshore Outsourcing

"Almost everyone who does a first offshore (job), has a problem, says Travis. The second and third attempts gradually get better. The culprit is often a bad process at home. Many IT managers have poor processes at their internal installations that manage to perform well even though the IT people may be operating in an ad hoc manner, he says. This situation, however, is a recipe for disaster when operations are moved offshore." From /Market Research Firm Touts Outsourcing's Cost-Savings/ http://www.techweb.com/wire/story/TWB20031031S0010, Duh Question: how much more cost-effective would it be to just fix your processes first? True story: A retail video rental company has an offshore team working on a new point-of-sale system. There are 70 programmers in India, and the project is scheduled to take 9 months opinion: fix your domestic processes and a team of 10 could build the same system in 6 months

permanent link

Cem Kaner at PNSQC 2003

On October 14 I had the opportunity to hear Cem Kaner talk at the Pacifc Northwest Software Quality Conference in Portland, Oregon. The title of his talk was How Many Lightbulbs Does It Take To Change A Tester? These are my notes and impressions from that talk.

Kaner's opening remarks focused on his views of where software testing has been and is going. After 20 years of practices, he said, little has changed, and the common practices have their problems. Compared to the programming community, testers have changed very little in their self-image.

One of the recommended practices (from Kaner's own work) is to only do black-box testing. Kaner question that and asked why it was considered good. What value is there in working without knowledge of the underlying code? The answer is that it allows the test, instead of learning about the product from the code, to learn about the product from the stakeholders and see how well the code does that. Tester gathers information the programmer didn't or couldn't consider. The more recent discovery is that sometimes the best source of that information is the code.

He then discussed the wisdom that black-box testing and external testing is necessary because "programmers can't catch their own bugs". Kaner drew a distinction between a private bug, one the programmer makes, sees, and fixes before anyone else sees it. Rate: 15-150 per 100 statements, and public bugs: all bugs left in code when programmer passes it to another person. Rate: 1-3 per 100 statements. In other words: 90% of the bugs are private, so programmers are actually pretty good at finding bugs. The insight here is that programmers find one kind of bugs, but it needs someone else to test other things to find the remaining bugs.

In the next part of his talk, Kaner discussed the view that testers and programmers should be kept independent. He brought up several areas where close collaboration actually is more beneficial than separation.

Para-functional Testing What used to be called Non-Functional testing, but Kaner discovered that too often mean "disfunctional testing". For good testing of things like security, usability, accessibility, supportability, localizability, interoperability, installability, performance, scalability, and the various other "ilities", testers must work with programmers because the stakeholders don't have rules.

Preventative testing. The best kind of preventative testing -- testing that prevents bugs before they happen, is Test-driven Development, said Kaner. It implies tight collaboration between the testing role and the programming role (sometimes so tight, they are the same person) Fast, efficient feedback is possible, and fast is good for learning. Communications cost is low.

Kaner's talk then covered the value of fast and efficient feedback in software development. Traditional external testing is say, 10 calendar days or more, plus all the processing, cycle time, said Kaner. In hours, cycle time for efficient companies, it's 4 hours for a simple defect. Up to 20 hours for a less-efficient companies. Filing lots of bugs is a form of denial of service attack to a programming team, because the processing time of dealing with the bug reports leaves no time to actually fix bugs.

The purpose of testing is to find bugs, right? Well, said Kaner, it depends. In discussing some of the points raised by the paper, A Manager's Guide to Evaluating Test Suites, Kaner examined how test results can guide a project manager in determining the status of the system. Do you want to find and eliminate the most bugs before a fixed date, Kaner asked, or do you want to develop an overview of the state of the system to allow the project manager to decide if it's ready to ship? One area where the testing is not done to find bugs, but to protect the company from risk of lawsuits -- do we test enough and well enough to accept the risk that our system may fail (and kill someone) but know we did the best we could?

Testers in Agile Teams

One of the facts of software development, said Kaner, is that pulling features out late will not save money. In the tradeoff between features, reliability, cost and time in a standard waterfall process, features and cost are used up by the end, so the only thing left is reliability vs. time. Thus testing is often compressed or cut. In evolutionary development where we have a limited set of features that work, then towards the end, tradeoff is do we have enough features to sell it or do we need to spend more time adding features?
Test then code Test-first development
Big Tests Up Front One test, one feature, iterate
The lesson for testing is that test code and testers must be able to cope with simple changes. This is where IEEE Std 829 Testing Practices is, according to Kaner, an industry worst practice.

A Final Note

Outsourcing, said Kaner, is a by-product of making the process rigid and "fully documented". It makes software into a manufacturing process, something that is easily outsourced. This is a fatally flawed approach for most commercial software.

permanent link

Wed, 08 Oct 2003

Code Reading

The process of taking computer source code and in some way coming to understand it. Comprehension and Visualisation of Object-Oriented Code for Inspections

Books

permanent link

Software MFA

Brian Marick reveals that he and Ralph Johnson are likely to begin a trial run of A Master of Fine Arts in Software at the University of Illinois in January. It's going to put into practice Richard Gabriel's ideas in software education.

I wonder if I could get in. As Brian writes, "Who wouldn't want to attend a program that people like Ward Cunningham or Dave Thomas or Andy Hunt or Martin Fowler or Mike Clark or Michael Feathers or Eric Evans or Paul Graham said was worth their time?"

permanent link

Tue, 23 Sep 2003

The pain of trying to get a simple question answered by a database.

Lately I've started to be really bugged by how dumb database constructs are. Let's take the table as and example. Sometimes would like a table to tell me how many rows it contains. Alas, the table doesn't know, it rudely ignores me. Long ago I discovered that table will only talk if SQL asks the question. So I learned to formulate a query to SQL that it passes on to table, and gives me back my answer. I don't like the middleman so much. I can't ask, "table, how many rows?". I have to tell SQL to ask table to count something. I don't know why, but I can't tell SQL to count the rows, I have to ask it to count either a column, or all columns. That's really dumb to me because, don't all rows in a table have the same columns? Why do I have to specify one column, or all columns (*), when I just want a count of rows?

Maybe I'm being too demanding, insisting that a table know about its contents. Tables are made up of columns. "Table, how many columns do you have?", I ask. Then I find out that table won't even tell me that. I have to ask this other table to tell me what's in the first table, and I still have to tell SQL how to ask the question for me.

Well, after a while I get tired of trying to get tables to talk to me. I try to get columns to talk to me, but they're even ruder than tables.

Then I have a little light bulb go on. Maybe instead of asking the tables, I can ask the content of the tables. After all, that's really what I'm usually interested in.

I know there are fifty states in the United States, but I forget what the capitals are. So I try that. I decide I'd like to ask, "States, what are you capitols?". We'll there's a table in my database that I'm told has all the states, but it turns out there's no way to ask the states anything. I can ask the table to tell me what's in it, but only by telling SQL to ask for something. I'm getting really frustrated about now. My SQL guide tries to be helpful by telling me it can find out for me how many states have capitols that start with the letter 'A', but I'm not interested in that kind of aggregation.

Sometimes my geography is bad, so I can't even point to the state that I want. I always get New Hampshire and Vermont mixed up (sorry about that). So I'd like to say "States, whichever one of you is Vermont, speak up and tell me your capitol". But again, I am back to asking SQL to ask the states table something about Vermont and it's capitol. SQL is really very picky about how I ask, too. Anyway, it turns out that the States table doesn't actually know anyway, all it knows is that there's another table that knows the names of all the cities, and can tell me if a city is the capitol. Talk about pointing fingers!

After all that, I finally figure out that I can tell SQL to ask the States table if any of its states are named "Vermont" and if there is one (which I already know exists) to ask the cities table for all the cities in Vermont which are its capitol -- even though I know a state can only have one capitol. What a struggle. All I really wanted to ask was "What's the capitol of Vermont?", and after several false starts I discover I have to ask a whole bunch of different middlemen about all kinds of things I don't care about and narrow down the results to the one thing I do care about. What a waste!

permanent link

Tue, 29 Jul 2003

Why Web Services Matter

COM(+)

CORBA

EJB

Web Services

Words of Wisdom

"This notion of a specific contract is crucial to COM and to component software in general. Without "ironclad" contracts, it would be impossible to interchange components."

Distinguish between the public interface of a module, which can change as long as it's a within-team usage, and the published interface, which must be immutable and adhere to the contract.

"reuse [is] undervalued and insufficiently rewarded by the sponsoring organizations."

"Services are grouped into coherent, contractual units called ''interfaces''."

"By focusing on the notion of a contract, the service perspective introduces an important distinction between the specification of a component (what it does) and its implementation and executable forms (how it does it). This distinction is fundamental to the management of dependencies between components and begins to address the important requirement to be able to replace a component with minimal impact on the consumer, often referred to as "plug-and-play". "

Create detailed, well-defined interfaces. Java supports the interface explicitly.

Design-by-contract

When evolving the Component API from release to release, do not break existing Clients. API changes must not invalidate formerly legal client code.

Prevention steps for clients (callers)

  1. Make method preconditions as strong as possible
  2. Make method postconditions as weak as possible.
  3. Ensure correct field invariants.
  4. Settle on and do not change class/interface "gender"
  5. Prefer creating a new interface to changing an existing one, even if the change does not break compatibility.
  6. Define checked exception semantics carefully.
  7. Publish (outside the team) APIs parsimoniously.
Evolutionary Changes
  1. Deprecate and Forward
  2. Create a new package
  3. Any new methods that add/remove params, overload and call the old methods.
Communicating Changes
  1. Clarify how APIs work when the spec is imprecise
  2. Fix Broken Windows
  3. Fix '''only''' Broken Windows
  4. Note incompatible changes in the Release Notes

Component Infrastructure

Articles

Evolving Java APIs

Component Technologies

COM

CORBA

EJB

XPCOM

permanent link

Mon, 21 Jul 2003

Use Cases vs. Stories

Summary: Use Cases can be any size/complexity, and are generally bound by feature set. User Stories are time-bounded and task-oriented.

The Power of Stories

permanent link

DefectReporting

Use a Tracking tool

Post-it notes, programmer's memories, even email, are difficult to coordinate on a team of more than 2 people working together for more than a few days. The tool doesn't have to be sophisticated -- it can be a spreadsheet -- but it must be shared.

Automated TODO list

Each item the bug tracking system is assigned to one person and has a priority associated with it. The list of bugs becomes a personal, automatically updated and prioritized, TODO list.

Document Work

The bug report has a life and grows with changes. Documenting that aproblem was worked on and resolved represents a knowledge base.

Coordinate Efforts

Untracked bugs can lead to wasted or duplicate effort. The wrong programmer attempts to fix it or multiple programmers all work on different solutions at the same time without coordination. Or a programmer works on an already-fixed bug not knowing that the fix is in. The bug isn't really a bug, it's a new piece of functionality, leading to scope creep. Low priority, easy, or "fun" bugs get fixed before the higher priority bugs. Assign bugs to one person at a time, and give that individual responsibility for resolving it or assigning it elsewhere.

Differentiate Priority and Severity

The affect of a bug on the software does not automatically correlate with the priority for fixing it. A severe bug that crashes the software only once in a blue moon for 1% of the users is lower priority than a mishandled error condition resulting in the need to re-enter a portion of the input for every user every time. Track the two separately, then triage appropriately. It helps to have input from others on the team on priority.

A Bugs Life Cycle

A bug can only be assigned to one person at a time. That bug may flow back to the reporter with a misdiagnosis or fix that doesn't work. The assignee may assign the bug to another programmer more familiar with the fault or for review. It may go back and forth from reporter to programmer a few times before it is fixed and resolved.

Fixed is not Closed

Once the programmer has made code changes and checked in the results, that programmer can mark the bug fixed, but cannot close it. The user who entered the bug is responsible for verifying and closing the bug if it really is fixed. The resolution of a bug should be different from the status. "Will not fix" or "Unable to reproduce" are among the other resolutions besides fixed that can close a bug.

Incidents and Faults

It should be a given that the same bug can cause different apparent problems to different users. It is also possible that a single reported problem may be traceable of multiple different faults. Therefore, several reported failures that arise out of the same fault should be tracked together. Consider splitting a single bug with multiple causes into a bug for each fault, and relate them as above.

Reviews Before Big Changes

Sometimes a bug or group of related bugs will result in a change that affects large sections of the codebase, or significant changes to one component. In these cases, before the programmer commits the changes to the team source repository, assign the bug report(s) to a reviewer, typically a team lead, senior programmer, or module owner. The reviewer can examine the proposed fix for compatibility, suitablity, and overall conformance with direction of development. Reviewers should be timely in their responses, and reassign the reviewed bug back to the programmer with approval or suggestions for changes. Once reviewed, the programmer should go ahead and check in the fix.

References

permanent link

Mon, 14 Apr 2003

Reporting Is A Code Smell

Reporting Frameworks Reporting is a code smell because:
  1. It's an indicator of incomplete or vague user needs. Essentially, a request for a report means "Gather a bunch of data and show it to me nicely formatted". Fine, but what exactly do all the pretty rows and columns tell you? Discover the true "why" for the report, and the need for the report evaporates.
  2. Reporting tools often cast domain objects in the role of dumb data containers. Instead of having useful business-oriented behavior, classes that otherwise would define suitable behaviors are diminished to just carrying around values from one location (usually an RDBMS) to another location, a piece of paper.
  3. There is a certain implication that the reports, once defined and written, never change and will always be what is necessary. Stories of reports that continue for years being generated and sent to people who never use them or even know why they are getting them are not uncommon.
  4. The typical reporting run, being a batch-and-queue process, can wreak havoc with the functioning an interactive system while the reporting is occurring. Reports against databases often do expensive table scans, killing performance for users doing transactional work. They exercise the object model in ways that are different enough from live transactional usage, and the tradeoffs between what's right for reporting and what's right for interactive are difficult to resolve.
  5. The request for a report is often a sign of an implicit idea that the computer can only do "data processing", and that the real analysis can only be done by hand. While that is still true for many classes of problems, the ability of programmers and systems to simulate and analyse is well advanced.
  6. Reports are often artifacts of times before the user interface hardware and software had reasonable formatting capabilities. Low-resolution screens and the limited ability of a screen print to a low-quality local printer, for example, compared to the capabilities of generating formatted output to a centralized quality line printer. With modern user interfaces and hardware, the information is often presentable on the visual, and if not, printing from the application in suitably formatted and high-quality manner is quite easy.

permanent link