My Worst Jobs, Part 3 - SALSA

Jack’s running log on how a $100 million a year software company spent $30-50 million on a software application that never really worked, and appeared two years late. We were to create a new database program, based on the theories of Dr. K_______. The database uses semantic object modeling to enable users who know their business to create complex databases without understanding database theory--things like data normalization, many to many relations, entity-relationship models and diagrams. In short, anyone can create a database. It was a nice theory.

Dr. K______ utilized a lot of the current theory of learning to create the semantic object model. By the time I started the burn rate for the project had to be around $2 million a month. I was spending over $1.5 million a year for testers, and more for writers and testing contractors. By the time it was all over I had run tech support, testing, a four person group of writers, a two person IT department, a testing tools group, a gaggle of testing contractors that varied between 2 and 15 people, and probably some other things I don't even remember. I didn't have any trouble managing anything but the testing.

The challenges of testing were the worst--because the application really never fully worked until shortly before it shipped. Frequently, QA/Testing became the scapegoats. We were forced to test things that wouldn't even work for a year. I was the nay-sayer. Eventually, the President of the company offered to put the team on a paid furlough to prevent us from piling up even greater numbers. There were times when we had 3,000 open bugs. If the app. had actually worked, we might have had 10,000 open. Although the database should have been able to handle the most complex data scenarios, we spent a large time just testing a database that held twenty city and state names. We couldn't even get to the the really crazy stuff.

Testing databases is extremely complex and requires a lot of automation and endless permutations of managing the data. By the time the app. shipped, we had only touched the possibilities. We started keeping a log of the various steps management took to get the project back on the rails. By the time I started work, they were already about one year down the wrong road, and were massively re-engineering the very guts of the system. I didn't spend much time editing what follows. In some ways it was just too painful. . .


1. I start at Wall on January 2, 1994 (jack). The project is already late. It looks about five times larger than anyone is admitting. I only discover on my first day at work that everyone except the testers are completely delusional about the state of the project. And even the testers are looking on the sunny side. I am stunned and amazed. On my first day, Dr. K______ asked me to weasel in and find out "what is going on in the PQA group."
2. Dr. K____________ is deposed in favor of E.G., a wall top Sales type who is now VP of the SALSA business unit. March 1994. By this date, I could already see the wheels coming off. Eileen is truly scary because she doesn't have a clue about software. Not even a hunt of a clue.
3. People are desperate for news of progress. Or even rumors of progress. Therefore, a Feature Complete projection was announced by by E.G. on 7/6/94: The Big Date-- Aug. 22, 1994. After she met with the Dev. Director for six hours, she came up with that date. We are feature complete, but we haven’t spec’d out or written large parts of the software yet. . . There is no way anyone can even grasp how much there is to do, how many dead ends and bottlenecks we have yet to face. . .
3a. I discover that they really didn't hire me to run technical support. They hired me to take over the testing department from the two competent people already running it. Competent, but not wise politically. They had lost credibility and needed a new leader to regain their lost ground.
4. So...eight months later, we weren't even close. If people realized they would be in crunch mode for the next 16 months, there would have surely been a mass exodus. But, alas, this was a cass of mass delusion.
4a. E.G. email of August 23, 1994: “Well, we didn't make our feature complete date and I didn't want it to go by without some acknowlegement. Well, so, we didn't make it. Ho hum. It would have been nice - and no one is off the hook - but we are absolutely moving forward. Do you realize that as little as 2 months ago we were in a week for week slip on identified tasks AND new tasks were somehow creeping into the schedule??? It was NOT a pretty picture. Can you appreciate the FABULOUS turnaround that the Dev. Director and his people have executed?? That we will likely deliver feature complete within spitting distance of our original target date? This is a @$%#@!!! miracle. HATS OFF TO THE DEV DIRECTOR AND HIS TEAM. KEEP ON KEEPING ON!!! Let's go for Feature Complete before Labor Day Weekend!!!” [more delusion!!]
5. UI Committee formed. To mediate and decide all UI issues. The U.I. is a complete mess. And why wouldn't it be the way the app. was being changes constantly.
6. "Comprehensive" Build Notes established 8/20/94 to assuage heavy QA complaints about code thrown over the wall with no notice or instruction. There was never a day on which at least one major part of the app. was not broken--scripting, generating databases, the U.I.--you name it.
7. The Fezes: help developers remember they are for end users. Cheap fezes given to all developers. Every time they see the fez, they were to remember they were doing this app. for "the people."
8. Retargeted product from app developers to power users. Major shift! Originally this was intended for serious databases. VP and Marketing manager decided power users for our marketing target were anyone who could use Word or Excel.
9. Company meetings every morning at 9:30
10. Cute names: for development groups and modules: Snoopy, Lucy, The Pez Dispenser, The Sous-chef, Cooking, Frogs, Runts
11. Development Leads, and more. Longest employed Developers become leads.
12. Morning company meetings peter out this week (Oct. 94).
13. Crunch Bars handed out to cement crunch mode. Can't eat them until crunch is over.
14. Boxes of Captain Crunch cereal handed out to memorialize crunch mode. Jan. 25 1995 [the app will finally ship March 1996] . They expect it will be out the door in three months.
15. [The Month of the Long Knives]: LH, DH, and SM all "leave for other pursuits" or something quite like it!
16. Quiet Time I 2-4 daily February 2, 1995. Let the developers focus!
17. Hand out four day vacation certificates to selected developers (2 days of which were weekends) note: they couldn't actually take the vacations until one year later.
18. A new batch of ca-ca will hit the fan tommorrow. Feb. 5, 1995: Email warmup: "WHICH PART OF MY MESSAGE WAS NOT CLEAR??. . . .we are going to have to pull some kind of god-@#$%!!! miracle to get the BETA out the door this month. . .what the hell is going on here? . . . .THIS IS SERIOUS. IS ANYONE CONFUSED??? I'll see you all at 9:30 tomorrow morning to figure out IF we can avoid losing the patient on the operating table..."
18a. quiet time abandoned.
19. The Sixteen Minute Olympic Motivational Address, Delivered by E.G., 9:30 AM, February 6, 1995, Seattle, Washington “WHERE IN THE FUCKING HELL WAS PQA AND THE OTHER FIFTEEN FUCKING DEVELOPERS?" ". . .I will NEVER NEVER ever let that happen to me again!" "I refuse to lose again!" “When I lost the Olympic Time Trials, I was one tenth of second from being in the Olympics. I vowed at that moment I would never lose again. "If anyone, anyone in this room, gets in the way of me succeeding this time, I will rip your fucking throats out..."
20. The Bridge: A group of people in a room, a war room so to speak, to "To tie up the loose ends." as if that's all that is left. Loose ends!!! This is a room with five of us “making decisions to tie up the product with a bow, “ as Dr. K_________ says.
21. RGB SALSA Golf Shirts passed out by The Marketing Director. She also talks a lot about green crystals. . .and the detergent that didn’t sell until they put in these magic green crystals that did nothing. Is this app such a total fraud that we need the crystals??? All the marketing focus is on green crystals. Eventually they roll out a web site that cost $1 million. For all the green crystals, you can't even tell what our product was actually supposed to do. They hired a famous cartoonist to create weekly panels around some gibberish about a chili, a monkey, and some other riddles.
22. Massage give away contest for best 2 developers/ 2 PQA for the week.
23. Marketing Hairy Legs, Underarms, ???, and Face contest. Marketing won't shave until we ship! Armpits and legs shown. (this gimmick lasted maybe two weeks).
24. Marketing Pie Throwing Event. As a reward some of the individual contributors get to throw cream pies at the marketing people. Glen does his Cagney thing with a pie. Finally gets to work out on someone in marketing for all their haranguing MIS. He has to be pulled off mashing their faces into the pie.
25. THE SCOOP: issues 1, 2, 3, 4 and 5. Neato features like interviews with developers talking about their star wars collections etc.
26. The Enhancement Committee. Bottleneck in the making. Decide all feature requests and changes. Later dropped in favor of another committee that did the same thing, but with a different name and more middle managers on it, which was later dropped for another committee, and on and on.
27. "Feature Complete" March 7 1995. Nyuk nyuk. A complete and utter fraud. Nothing is complete or even close.
28. Feature Complete T Shirts handed out. Most of the testers give theirs to street people outside our door in Pioneer Square. Once in a while, over the next year, you'd see a bum on the street wearing a SALSA feature complete t-shirt.
29. Dr. K_______ e-mail, “I Still Don’t Have Two Tires” In short, the app doesn’t work well enough to track a pair of tires in a tire shop. On one hand, David says we’re almost done, and then he says we can’t even keep track of two tires! He finally believes us. It doesn't work. I think Dr. K. could finally track two tires six months later.
30. Marketing is no longer allowed to do anything "Fun" until we ship per E.G. Feb 23 95.
31. Work 80 hours a week, a few weeks culminating in BETA ONE ….. beta one cannot hold 100 records in two unlinked tables. Who're we fooling?
32. Champagne Party as we ship beta one. Massive drunk. People are totally burned out.
33. PQA/PD/Tech Pubs get three days "off." Two of the days are Saturday and Sunday.
34. The Friends of SALSA Program instituted (five users who provide feedback, only one did) as an attempt at usability testing. We also did a $50,000 usability test in Texas that yielded bupkus. Our 50K did yield a couple of suggestions.
35. Friends of SALSA tee-shirts are distributed. To kick off the program (which of course petered out after a week).
36. The Dancing SSSs. February 1995. "add some marketing sizzle to the product" by introducing an animated splash screen that caused the app to crash for weeks, slowed down loading by 70%
37. The Color Wars: incentives for fixing failures. Split dev teams into colors. Development teams earn prizes! [this is the first in a disastrous set of giveaways to developers for doing their jobs].
38. Pass out embroidered hats for the Color Wars Teams
39. Eileen purchases a $250 Gong, to be bonged when we ship, which she thinks is very soon.
40. PQA Cool Developer of the week award—whoever did something cool, or screwed up the least or fixed the most irritating bug.
41. The Scoop publishes bogus interview with Jack Brummet.
42. PQA Cool PQA guy of the week award. Whoever in QA pulled off something outrageous gets a CD.
43. March 13, 1995 - pend everything in sight - for beta 2:::hide the bugs. Pend 500 bugs.
44. Job switch: "Promote" CW from Tech Pubs Mgr. to Usability Manager
45. Job Switch: Put John Brummet in pro tem charge of Tech Pubs and QA and Tech support.
46. Job Switch: Put The Marketing Director as pro tem in charge of Technical Support so Jack can run tech pubs.
47. Job switch: Dr. K_______ becomes a tester. "Now I understand why you guys are so cynical. Nothing that they say works, does."
48. Death of THE SCOOP. We'll really miss those developer profiles and their favorite foods.
49. Someone brought in a Cupid today—a guy in a costume, running around throwing candy and dumping his cigar ashes in JH's trash can, saying, “If it don’t hurt, it ain’t love.” JH threatened to brain him.
50. The Learning Tools. Some usability attempt (cue cards etc). A whole suite of stuff to help people learn. More feature creep. And yeah--we were supposed to not need these tools because the app. was so intuitive.
51. Quiet Time II. Oh yeah, the first one fizzled out, we forgot to document its death.
52. Weekly award presentation to a developer: “SALSA Superstar Award,” along with cash or dinner
53. The Failed Verfication War - heavy QA/PD tension. QA tracks fixes that dont' work.
54. The software isn't ready to beta again until all help links work - according to the Business Unit Head and the CEO/Chairman. What about actually work? The help will work, but the app will not.
55. Serial Testing Cycles. QA is ordered by VP Mall to test the whole product every week! As if we could. "Top to bottom in a week." But it still crashes in the middle. Dozens of open bugs on just using city-state records.
56. Job Switch (or, her third demotion): "Move" CW from Usability to be in charge of Learning Tools
57. Job Switch: Put Jim Mothershead in charge of Usability
58. EG/The Business Unit Head: "We don't believe in face time." But we bring in breakfast now too. We just don't publicly believe in face time.
59. Mardi Gras/Beta 2 Party. Depressing.
60. Six hour meeting to decide Beta "2.1" date. Jack. The Dev. Director and Business Unit Head hold a tension-filled meeting. Jack’s last statement: “we're not even ready to do a Beta 1, we haven’t even done beta 2. . .why focus on beta 2.1?? the app doesn’t work and there are things in here that may be technically impossible. We are wasting time, money and credibility. Let’s fix it. We're technically not even at alpha yet"
60a. The Vice President: “It’s not let’s fix it, Jack, it’s let’s ship it.” Jack's suggestion we are not at alpa even is shouted down.
61. End of Serial Testing Cycles (product too unstable to use for a week, or ten minutes at a time). Management finally agrees with QA again. The serial testing pass has yielded two thousand bugs in 9 days, the opposite of what they hoped would happen.
62. The Chairman/CEO visits The Bridge and has CW do a demo. He freaks out. Lambeth is trapped in there. Writes concurrently in email to PQA. "It's so f****g tense in here. The Chairman grilling CW...oh man."
63. Jim L. Emails a few minutes later "I figured out how to get through this. I just look over and picture them naked."
64. Beta Two nightmare. Jack halts shipment when the app stumbles on a simple problem. (CW shrieking "Works on MY Machine!" and "LET'S GO TO THE AIRPORT") Jack has a vision of taking the Louisville Slugger to her head. And the room exploded in a crimson fusillade.
65. SALSA Sweepstakes for new terminology (Lambeth won 2 sets of dinner for two for a load of terms that were all eventually abandoned in favor of newer terms purchased by marketing wizards)
66. Shipped Beta 2 with completely untested new QuadBase database at the last minute. Jack had a little note at the end of the read me, warning people not to save anything important in their database, because surely it will disappear if they actually succeed in creating one in the first place due to “transient anomalies.”
67. Ship Beta 2.1 because The Business Unit Head thought it was too long between betas. Nothing works better and some things are much worse. Bon appetit customers!
68. The Insulter Application--> a SALSA app that runs. Mostly.
69. End of Development Leads. They were ineffective.
70. SALSA TOO SEXY T Shirts appear. WHAT DOES THIS MEAN?
71. Movie/dinner/Pike Place Market Spa gift certificates to various parties.
72. The Dev. Director and Jack hold a two hour meeting with The President on process.
73. Beta Three. App starting to work.
74. Drinks and chips at Irish bar, awards handed out, tee-shirts (SALSA: too sexy) handed out, pinatas broken, and some cash awards are disributed.
75. Regular re-supply of Doritos for CO and Mints for MW.
76. The Bonus Pool/Stock Options. "Oh boy we're gonna be rich." If the app sells $10 million, we get two mill to split up amongst 24 of us.
77. End of The Bridge. The Dir. of Development, Danny, Jim L, and CW return to their former offices.
78. The Chairman and CEO stops by Jack and Don's office for a chat on the product. Comments bewilderedly on Jack's artwork. He is angry and worried. Befuddled about how a project could be so far off course.
79. Drinks for everyone at Iguana's to celebrate the beta we shipped today. Jack note: no one installed the last beta, except The CEO/CHairman.
80. The Runtime Re-architecture and Code Review. A complete rewrite of one of the most critical app elements.
81. Beta Four. A lot of help stuff.
82. Quiet Time III. Because II fizzled out.
83. Dedicated Runtime Test Development: linkmany, xtable suites. Unit testing two years too late. I have two testers focused on this, plus a comprehensive automated test suite.
84. The Ed Report. QA member tells what actually works and doesn't. Like it or not. It is extremely depressing because mostly what works is installing it. After that, you're in deep ca-ca.
85. Script/formulas/PERPS Specification and Design Review. They are about to reengineer one of the core pieces of out technology. These pieces have a waterfall affect on almost everything else in the program.
86. Development Leads II. They bring back the leads. New Ones. So now we have the doomed new leads leading the old resentful leads.
87. Lobster Dinner for the developers (but they didn't get any wine!). What is that about? Lobster and coke?
88. Hiring sales crew. Why??????????? It’s a year early, I think. SO now we have all these sales people demanding that we just ship it.
89. The business Unit Head plans to hire 2 or more new development managers (will solve all our problems).
90. Developers writing code in their underwear (not official gimmick, but duly noted with much humor) . One guy acttually.
91. A.R. and his next gen dev. group roll out plan to ship 32 bit version with 60 days of SALSA FCS (note: it had not shipped 9 months later).
92. Coverage Analysis for The Business Unit Head: somehow prove the product is ready to go! When I gave it to her, it showed we were able to cover 12% of the app. We were blocked from the other 88% because it either didn't exist, or we were blocked from accessing it by bugs.
93. They take us to the Ziggy Marley show::::copious drinks, surreptitious bong hits, reggae.
94. Recruit J. Mothershead from PQA to write Import. An extremely important piece they left 'til very late. And they had no one to do it, do they used our testing tools developer.
95. Getting serious about tech pubs dept. /Hiring Kieran Mahoney (whew!) Jack is liberated from running Tech Pubs.
96. The Dev. Director and Jack hold a four hour meeting with VP of Dev. on process, what to cut, how to get code into testing faster.
97. Drive toward a new ship date. This is our 12th new ship date.
98. Providing Starbucks coffee only to SALSA group; rest of Wall Data gets Seattle’s Best.
99. The Business Unit Head takes Polaroid pictures of everyone day of Beta One ship, then buying black scrapbooks to make album, in which folks made numerous questionable sex and drug references in captions.
100. Writing a script specification in September, 1995. Too late. Oops the code is done. Oops. We have to rewrite the entire scripting module.
101. PD/PQA/Tech Pubs move to the Seventh floor
102. Pick-your-own-Ikea-lamp
103. You get to pick your own plants.
104. SALSA Story CD ROM
105. shifting around officemates. "Seeing if we can stir up synergy."
106. SALSA watches passed out. Jack's falls apart on Day Two.
107. 2nd hard drives for PD a year too late.
108. Picking your own plants cancelled. Too expensive.
109. The Marketing Director recently offered a bottle of “REALLY good” red wine for something. For naming some part of the app.
110. PD/PQA at Analysts’ Conference; purple polos and new pants/shoes for Danny and Ed
111. Cookies once a week. An incentive to show up?
112. Meatloaf and noodle surprise: free food three nights, and one Saturday a week. All the bad food you want.
113. If we get the product out by Thanksgiving, we get thanksgiving, Friday, and the weekend "off."
114. Cookies every other week now.
115. one month bug fix challenge:$2,500 potential incentive per developer
116. Death of the Ed Report. Too depressing.
117. Problem Finalization Board. This gimmick ran until the end.
118. Dr. K_______ becomes a tester again. To prove PQA is slacking, and being diffficult. And, naturally, he totally freaks out.
119. We lose the database shootout: We get second place--the steak knives!
120. CW deposed (demotion four) "temporarily" as head of Learning Tools Group
121. Kroenke runs the learning tools group
122. Ending the wine club. Two developers had been having a glass of wine with dinner. The Business Unit Head did not like that.
123. Starter Kits: The latest gimmick. Prebuilt applications for people to use. We can build them around the limitations of our database.
124. Do verifications faster, you lazy PQA animals. Verifications are screwing everything up. "We don't know where we are unless things are verified."
125. Bad Models--> The Business Unit Head "we must save our users from ever making one, and prevent SALSA from incorrectly allowing or using one."
126. Bad Models--> Dr. K_________: "there are bad models." Looks like a fight over bad models. Allowing bad models means we are not a user-friendly app.
127. Visual Freeze 11/3/95
128. New gimmick. The Doctor leaves the ivory tower. He had a sweet office sweet/think tank right on the pier, above Pier 56. email: "Dr. K_________ On The Move" (moves back to 1011 Western) Oct. 1, 1995
129. All failures that prevent testing must be marked as BLOCKING in Database. They must be emailed daily. They must be written on the whiteboard. We have now marked over 350 bugs as blocking.
129a. Developers and leads are now massively marking bugs as "cannot reproduce." Hundreds. Why? Because the app. crashes now before you can get to those bugs. Therefore, NO REPRO!!! Jack massively kicks back four hundred of these one night, igniting another anti-testing firestorm.
130. The Punch List (checklist of everything yet to do). Because they are in denial about the bugs....the focus is on what we need, not what's wrong.
131. the SALSA launch kickoff meeting/cocktail hour for mgrs/leads. Weird meeting weird vibes, Kroenke tears in eyes about his dream.
132. Death of the Dancing SSS's. Killed by The VP of Dev. Yeah!
133. Beta 4.1. Includes Import. Sort of.
134. The Business Unit Head forces Jack to send out letter to beta sites apologizing for the arcane import instructions he included because no one else wrote documentation.
135. The Business Unit Head hands out black SALSA ballpoint Parker pens to celebrate Beta 4.1. I think we're about at alpha now...almost feature complete.
136. just ship it stickers/if it ain't broke don't fix it. - Tech Pubs Gimmick
137. Tying specific areas up with a bow: "ready to ship." The VP of Dev.
138. Find Goodness. The VP of Dev. Our new watchword.
139. "NO MORE BETAS" Says the Chairman/CEO: "You have embarrassed us!" to the Business Unit Head !" WALL DATA DOES NOT SEND OUT F*****G BETAS THAT DO NOT WORK!!!!!! A**HOLES."
140. Dr. K__________ Curbside Service. . .new gimmick to help developers. The guru will go to their desk to help them figure things out.
141. The VP of Dev steps in & takes some control. Not quite a takeover, but he clears everything now. In a lot of ways, he does help save us from ourselves. He takes decision making away from our business unit head. Obviously the new Dev Dir. and the Business Unit Head are on their way out the door. Sooner or Later.
142. Bad Models per Dr. K__________: "If they make bad models, they have to suffer the consequences." Jack said “But I thought we were supposed to help them NOT make bad semantic object models.”
143. PQA Contractor October/November/December Test Pass. Jack has 23 testers on the app. now.
144. The VP of Dev. reiterates in a feature meeting: JUST SAY NO TO NEW FEATURES!!!
145. PQA Coverage & Closure Matrix
146. Feature Complete 2.0 (11/3/95) but masks, grids, queries, medium sized apps, larger data sets, scripted views, don't yet work at all, or close to correctly. So we're not really even at alpha, but are alleged to be at
147. PQA "Ten Most Wanted" white board ordered by the VP of Dev, and Business Unit Head. Every day we propose the worst ten bugs.
148. The Dev. Director proposes $5 bug fix bounty.
149. Bug Bounty 11/26-- each GPF and assert a developer fixes is worth $10.
150. Spin control: Dr. K______ spin control. "John - Ever hear of the sandwich technique? -- give them a complement (slice of bread) , then the criticism (the meat), then another complement (more bread). It's supposed to begin and end on a good note, while getting your message across. It might be worth a try!" So QA is now supposed to say something nice in every bug report. . .before they slip in the meat!!
151. Taking pictures of all employees and posting in kitchen, then abandoned that idea, leaving blue posterboard with about 1/3 of the pictures ripped off b/c people quit or were fired and it looked so damn depressing.
152. Scapegoating "they [pqa] don't work nearly as much as PD" "they've been putting in a lot of fluff and bullshit bugs" "they haven't covered undo/redo at all" Note: QA is 2,000 bugs “ahead.” 153. Elaine (New salesperson) sending Power Bars for all. She shows Jack and Sam her nipple rings after a meeting.
154. Problem Assignment Board swells from two to four, and sometimes six. Different from Product Finalization Board.
155. Sales Team hosting cocktails at Cajun Corner (much "just ship it warts and all" talk)
156. The VP of Dev. is matching the bug bounty now. It's up to $20.
157. New plan by the VP of Dev. and the Dev. Director. Fix the worst 500 bugs and, then ship. Wait didn’t we do this 26 times before, and we’re up to 13,000 fixed? (really). They did not understand goodness or how it emerges, but said “If the 500 worst are gone, most others will be too and that us goodness.” Jack says fixing the worst 500 bugs will take six months, and may not get us to goodness at all. In fact, fixing 500 bugs will most likely expose 500 more. Or more.
158. PQA don't cover the whole app. You test too much. --just make a final pass using starter kits. "Quit finding bugs"--says The President, "You have to cut it off sometime or you'll find bugs forever." The President: “You manage the problem by managing the testers. You’ll never ship if you just keep testing full throttle. Look for the goodness now.”
159. Announcements of first units sold by Sales Team. We can barely run these apps with SALSA right now.
160. Dev. Director and Jack now report to VP of Dev. In a lot of ways that is a relief. I can actually reason with him. But it is hard to reason with people watching $2 million a month go up in smoke with no visible progress.
161. Product stability at all costs. Again, VP of Dev. mantra, “QA: look for disasters and focus on goodness. Be positive. We only want to know about disasters. Find the goodness!!"
162. Hard code freeze begun week of 12/18. All check ins must be signed off by a code review and a PQA unit test.
163. Shipment of Press beta + to other sites + to jim simpson
164. Eileen: Pay PQA $20 for every GPF/data loss found (announced by Business Unit Head at Co. meeting) . PQA immediately enter over 100 data loss bugs.
165. VP of Dev. quashes above payment program as foolish and insane after coughing up the two grand.
166. VP of Dev. incents pqa to reduce verifications (there are 500 bugs we can’t check because so much else is broken) and complete test coverage matrix ($500 each for tester non-managers) .
167. Fix the bugs at hand, then make a call. Report verif. failures and regressions only. (12/22) .
168. Fix Top 50 failures per day
169. Roll out FOURTEEN DAYS: a plan for testing and fixing leading to shipment 12/16. I think not. Maybe march.
170. Assign all new failures to Dev. Director. Only Fix failures from "Dev. Director's Bucket." We are now resorting bugs a new new way. Because?
171. Bizarre email from Business Unit Head. "Time to Shine" - Include up to 10 names and addresses of friends and family for inclusion in our direct marketing campaign. These people will receive the SALSA Story CD-ROM plus follow-up postcards, mailers,etc . YOUR FAMILY AND FRIENDS WILL FINALLY KNOW WHAT YOU DO! I am too ashamed to put any names in.
172. FOURTEEN DAYS plan for testing and fixing is abandoned 12/22. Fourteen days will not do it. Maybe fourteen weeks.
173. Begin testing again, but focus on bugs generated through using starter kits.
174. Unit testing and verifications in lieu of product testing. Only verify don’t test!!! Yeah, right. We keep testing and I keep getting heat.
175. Only failures Marked as STARTER KIT will be eligible for entry into the failure.database. Enormous testing impact. Now it takes hours to find bugs formerly found in minutes because we have to use the starter kits, which take up to an hour to generate in our molasses slow system. This helps slow down the bugs all right.
176. New failure prioritization scheme matrix: Priority/severity only fix up to P2 high. P2 medium close.
177. All non-verification testing must be related to starter kits.
178. Filming SALSA internal rollout video.
179. PD/PQA get Christmas Eve and New Year's Eve night "off." Xmas Eve was a Sunday.
180. Announcement of "Death March" to shipment to begin Dec. 31, 1995. 24 hour round-the-clock shifts, everyone "gets" to sleep 8 hours per day (VP of Dev.).
181. Announcement of the "Death March" to the rank and file will be postponed to at least 1/4/1996.
182. Death March again postponed, but no announcement made 1/4/1996.
183. "Dev. Director's Bucket" Abandoned, one week later 1/5/96. It was patently absurd.
184. NEW incentive program begun 1/5/96. One weekend.
185. Breakfast to “get together and enjoy ourselves.” DEev. Director and Jack hammered on in turn by Sales Head and Dir. of Marketing. Nasty personal attacks. Bad vibes all around.
186. New plan for finding failures 1/8/95: We start to test again, because Dr. K_________ finally listened and found the app wheezing. Now we can test for::: 1) gpfs, asserts, 2) data loss, 3) discretionary tests ---the testers know this app inside and out.
187. Dr. K_________ proposes a product enhancement, which he claims to the PFB (product finalization board) will increase the sales of SALSA by 30%. He dances around the issue that he has previously said the product must be nailed to the wall with no more changes.
188. Eileen reiterates once again "we don't believe in face time."
189. Weekend ending Jan. 14 -- another fix-it incentive program. Handing out more dough to developers. Where is The Goodness?
190. SALSA announced to the public Jan. 15.
191. Wall Data "internal rollout" of SALSA 1/16/-17. SALSA T-shirts and CDs.
192. PQA redouble its efforts to help PD finish (better reporting, less dupes, simpler cases, more and more cooperation) so they get off and PQA becomes the critical path.
193. Final complete coverage pass through the product using Jim Mothershead's beloved ECCO coverage matrix.
194. Ship Date announced internally to SALSA: February 8, 1996.
195. Problem assignment Team is no longer Danny and Ed, but Don and John.
196. The Saleswoman with the Nipple Rings brought in our FIRST SALSA ORDER from General Motors today (December 9, 1995). The Chairman/CEO sent an email: "Congratulations........We're on the way.........LET"S GO !!!!!!!!!!!!!!!"
197. The Freeze Committee is born (Jim M, Chris O, Alan R, Dev. Director) .
198. First Release Candidate is scheduled for February 2, 1996.
199. PD sponsored cookies now once a week again.
200. Everyone is measured for their SALSA letterman's jacket. A $200 leather sleeved thing.
201. PD/PQA /Tech Pubs get the weekend of Jan. 20, 21, 1996 "off."
202. Return of the gong.
203. Announcement of a "free" week "off."
204. Announcement of $1,000 net "vacation money" to be distributed ship night.
205. AM Freeze Committee meetings. We ,meet at 2AM every day. 4 am builds announced by Marvin
206. First attempt to produce Release Candidate: one solid week of chaos. Not one RC emerges.
207. Ship party scheduled for the week during which we ship, when we will all be on vacation.
208. February 7 THIS list was unveiled to Marvin Mall at a midnight chat..
209. Existence of another Top Secret PQA gimmick, the CYA log disclosed to VP of Dev. QA answers with charts and statistics every charge made against them in this finger pointing debacle.
210. Dr. K_________ re-arranges his office in order to pay undetected games of Doom while waiting for the next Release Candidate.
211. February 9, 1996. SALSA Ships. Almost two years in crunch mode for a lot of people. Given away: Champagne. New Dev. dir. hosts a party at a nearby sports bar. $1,000 cash each. One week + off. A leather trimmed SALSA letterman's jacket. Relief. We are spooked (QA).,
212. New Dev Dir. announces at champagne toast that he is in control now. VP of Dev. is done and he is in command.
213. We go home and off to Mexico, Arizona, Tukwilla, et cetera.
214. Big Ship Party 2/23/96 at 2118 1st Ave. Inebriation, fornication. A girl takes her top off. People are blind drunk. Jack and Ed kill a couple of bottles of $120 burgundy. People swill $80 bottle scotch etc.
215. Announcement that the bonus pool has been killed. No further rewards except being here. Alas, QA had 8% of two million bucks. Of course you’d have to be on heroin to think we’ll sell more than 100 copies…
216. Return to work. We all feel like skinned animals.
217. Free copy of SALSA to all employees. Playing of the SALSA rollout video, free t-shirt, and complimentary laminated SALSA Fact Sheet.
218. Post Feb. 9, 1996: Announcment by of SALSA 1.01. Announcment of SALSA 1.02. Announcment of SALSA 1.02a. Announcment of SALSA Zappa (2.0) ==about two months after 1.0. Announcment of SALSA 2.2. Announcment if Joplin. Salsa 2. something. Announcment of Morrison. Salsa 2. something. Announcment of Lennon and Lennon II. Salsa 2. something.
219. Jack leaves Wall Data. Salsa.
227. Salsa would work fairly well within a year or so. But it stiffed in the marketplace.