When a long distance runner has been running long enough that endorphins kick in, said person experiences a feeling of euphoria that, literally, marathon runners have known to become almost "addicted" to. I am wondering if there is not a similar sense that programmers get when they are in such a state of "flow". That is - when progress is coming quickly, providing a positive feedback cycle to the developer. Each sprint of code coloring multiple tests green, nailing down 8 hours of work in two. Each feedback cycle providing a sense of accomplishment normally reserved for, perhaps, gaining a level in World of Warcraft :) (I kid... kinda)
The purpose of the people to whom the developers report - be it an application development manager, a project manager, or someone else, in my view, is to make sure developers stay "in flow" so they can, as frequently as possible, experience "programmer's high". What can drive programmers into this state:
Having a project with purpose
Nobody is going to get into a "programmers high" if the project they are working on, or the organization they are working for, is not compatible with their values. If you feel like you are doing evil (and assuming you are not evil yourself) - you will probably do as little as possible that you can get away with, which certainly wont keep you into the flow.
Now, not every project is going to involve optimizing the supply chain of anti-HIV drugs to poor African children, but it does not need to be that purposeful. Even writing the 99th version of a general ledger application can be purposeful if it tickles that "urge to invent" that resides, or at least should reside, in application developers. Any program that will truly have impactful results that, say, increase opportunities for yourself and your co-workers can be a candidate for "purposeful".
Having a boss and organization who you trust and respect (aka "no bozos")
Related to purpose, if the people or organization you report to make you miserable, you will have little means to get to "endorphins". You can't feel good if your efforts will only help people that, for whatever reason, you think don't deserve the result or won't appreciate the result. The biggest reason people leave their jobs is having a poor relationship with their direct supervisor. Not having a good relationship will make it impossible for your work under that supervisor to reach "programmer's high".
Ability to Avoid Interruption
Having a means to develop without being interrupted is critically important. While this is a common topic in much writing about programming, it bears repeating that yes, you need to allow developers to, if they can't at least have an office, have an environment where they can turn the outside world off for 4 or more hours at a time so bigger problems can be solved.
Great Tools and Techniques
Tools like ReSharper that allow for increased velocity in refactoring, tools like TFS and SVN that make source control less painful, and techniques like TDD that make change less risky - are all ways that help developers stay "in flow".
Smart Coworkers
Related to the no-bozos for your boss rule, having coworkers that collaborate with you rather than fight with you - ideally coworkers that you like and respect. Having to be "the hero" time and again to pick up the slack of others is a quick way to ruin motivation. On the other hand, having coworkers around that are also in flow tends to promote an culture of excellence and mutual support where the people involved can help eachother when they get stuck (ideally, in a forum that doesn't interrupt that second).
Recognition that Programming isn't Typing
While its related to the point above about having a "no-bozo" organization, the kinds of places where a programmer's high can take place tend to be the ones where "programming" isn't considered the act if typing C# code into an editor, but rather, a critical intellectual exercise where the developer creates abstract models that solve business problems. This, of course, requires recognition that software development is a creative exercise where that does not lend itself to draconian, heavyweight processes.
A Good QA Department
Critical to being able to be in the zone is having a complement in the Quality Assurance that knows how to do their job well, and who can constructivley work with software developers. That means, QA people who know how to put in re-creation criteria, who avoid interrupting developers but rather report issues in the correct channels so they can be prioritized, and who do not try to get into an adversarial relationship (one of the worst ideas ever is to hire one firm to do development and another to do QA - especially if the QA firm also sells developers and the QA person is incented to grow the account... just a thought).
That said, this is more likely when QA is seen as it's own valid occupation, such much like the "Editor-In-Chief".
Conclusion
When these things don't happen, you tend to have less productivity, more turnover, more acrimony, project failure, and competitive disadvantage to those organizations that "get it" and can harness the act of "getting developers in the zone". The job of every person who manages software developers is to get their people to "programmer's high" as much as possible.
A colleague of mine, and TFS guru, Dan Sniderman, is the inagural speaker at the newly formed Chicago alt.net user group. He will be speaking about what TFS and CI can do for your team. Afterwards, as usual, we will head downstairs to the bar area and have drinks as we discuss all things .net.
Details here.
It is high time that Business Intelligence get the benefits of the language "Cambrian Explosion" and agile revolution. Think about BI for a second. Most of the talk around BI is oriented around tools - a stack that ties together presentation, storage, and logic, all in the name of avoiding dealing with pesky programmers. To the point that "requires no writing code" becomes a feature point. How did we get here? And how do we get out?
In the olden days, if you wanted a report, leaving aside BI for the moment... you had to ask your IT department for the report. You were on their schedule, and more often than not, the backlog was very long. Leaving aside why this was the case (i.e.budget shortage, lack of IT/business alignment, etc.) - it was. This begat two primary developments, the shadow IT department, and the market for tools that empowered the business to, at least try, to generate their own reports independent of IT.
Now, in the intervening years, these two developments have not really stopped at all. There is still a ton of shadow IT, and a ton of tools that purport to help you generate business intelligence by using an integrated stack of tools that, in theory, allow BI to happen without programmers. The question is... is this a good thing?
I would say no. BI tools, more often than not, tie you to not only a platform, but frequently, a specific product. You can't take your BI developed in Microstrategy and run it in Cognos, at least not very easily. And it makes sense why - as each of these tools competes on the basis of capabilities, and there is therefore no motivation to port the capabilities of one BI product over to another. And because there is no obvious short term economic justification from the tool vendors point of view, it simply doesn't happen.
Of course, the medium to long term economic justification for tool vendors for this is very good indeed. By creating an ecosystem of BI that allows for greater innovation and better solutions, BI will receive much greater investment. The savvy players who take advantage of this will do really, really well, just like Microsoft prospered by having an open PC platform and Google prospered by having an open internet platform. What has to happen, however, is someone has to move first, and given the nature of the space - big corporate buyers - it has to be one of the big players to do it with any kind of credibility.
That said, it does not help that there has been little standards innovation in the world of SQL. Not to say that it doesn't happen, but lets put it this way - nobody is proposing SQL as a new .net language like they do for F#, Ruby, or even Boo. SQL is just now standardizing how objects work... and worst yet, the language continues to get balkanized - especially in BI land where extensions for doing cubes and other specialized functionality tend to differ from vendor to vendor.
So how do we untie this gordian knot and get to a place where BI is portable, testable, and exists in a manner that allows diversity in authoring tools, persistence mechanism, and presentation mechanism? I humbly submit that F# should be the language of BI.
Why F#? Well, functional programming in general is oriented towards the folding, summarization, reduction, and calculation of sets of information - that is - data. SQL is mostly a functional, declarative language anyway, so moving to F# as the lingua-franca of data should be a no-brainer. Imagine a world where BI is:
* Persistence Ignorant rather than Persistence Obsessed
* Portable from tool to tool - so long as it can parse F#
* BI authoring tools allow business users to use a GUI to write F# constructs rather than balkanized SQL constructs
* The benefits of a modern functional language (ASTs, automatic generalization, massive parallellization, etc.) are finally tools that are easily available to BI
* Allowed to have the benefits that the agile world has brought us (testability, etc.)
Imagine a world where you write a Domain Specific Language (DSL) in F#, and the BI tools manipulate the DSL. Imagine being being able to swap out different persistence mechanisms based on strict performance characteristics, rather than having to pay the port tax when you move from one persistence mechanism to another. Few people in the BI world have been exposed to the recent "Cambrian explosion" of new languages that have emerged in the last few years, and that's a shame, because some cross-pollenization would be very compelling for new kinds of solutions to emerge.
A recent Gartner CIO poll reported that CIOs must ‘Make the Difference’ by replacing generic IT with distinctive solutions that drive enterprise strategy. This means that true BI that differentiates will likely be invested in. It would be a shame if we continued to have all this BI live on vendor specific islands that were unable to leverage some of the state of the art work going on in computer science. On the other hand, BI that leverages these new capabilities that the computer scientists like Don Syme are giving us will have a great chance to "make the difference".
I conclude this with a call to action. If you are doing BI, ask why we are using the same basic language we were using 10 years ago. If you are a language geek or a software developer, ask why what you are doing, particularly if it generates information that is used in the strategic decision making process, isn't considered "BI". Whomever is the first tool vendor to get to this vision will probably get to have a great deal of control over how it gets done - and the field is very green at the moment for someone to fill this gap :)
No. Period. Full stop.
TDD guarantees quality about as much as religion guarantees morality. It is probably tempting to draw a cause effect relationship from one to the other, since current TDD practicioners tend to be very good software developers, but the idea that the TDD process invariably leads to quality is, frankly, a case of post hoc ergo propter hoc. Not too much unlike an assertion I heard many years ago in college that 98% of people who are in prison have drank alcohol - and therefore you should not drink alcohol (leaving out such statements of truth that 100% of people in prison have mothers... which could lead to the false assertion that you should avoid having a mother).
There is a ton of value that TDD brings us. TDD can:
* Help you design software
* Isolate and crystallize requirements
* Help you avoid writing YANGI code - since code that is not written to turn a test green probably does not need to exist
However, like many other things, it ain't everything. If you are inventing, say, the next mass produced hydrogen car, would you design each piece by having the mechanical engineer draft the dimensions of each part first? I am not convinced how TDD can help in the invention process, where you have a vague idea of what the inputs and outputs are going to be for the software component you are writing.
Said another way, during the design process, if I have to figure out a test suite for each iteration of a design that is rapidly changing from one iteration to the next, it can follow that TDD can be a drag on my design velocity. When I am writing "exploratory code", say, to implement IQueryable over some new thing, frankly, I am going to experiment until I get something very simple working at all. Same thing applies when I am doing work where I might be applying a somewhat novel algorithm from some researcher to some problem domain where it has not yet been used (sidebar: those, by the way, are REALLY good cases for something like the F# interactive window, but I digress).
Now, for very well understood domains where you are going from a user story on physical medium... a case where you have a pretty good understanding of what the vague shape of the end solution will be, TDD rocks! But the idea that TDD is a process that can, by nature of the superiority of TDD as a process, produce bug free code, is one hell of a stretch. That assertion, in my view, relies on the tautology that you write the tests (and this, I will be more broad and say unit, integration, and all other 'coded' and scripted tests that come from developers) - and that quality is measured by the same tests. In the land of code, it is circular reasoning, and ignores the fact that all creative processes produce something that someone other than the creator will regard as a defect. This is why even the greatest writers, from Stephen King, to Carl Sagan, to John the Baptist had human editors who inspected the work product, manually, and produced redlines bug reports. You still need an editor-in-chief.
In other words, TDD is great, but as a process, it does not lead to bug free code.
Update, with example:
Given this:
[TestMethod]
public void SillyAssertion()
{
specialInt fortyTwo = new specialInt() { Value = 42 };
Assert.IsTrue(fortyTwo == "Answer to Life, the Universe, and Everything");
}
And this:
public struct specialInt
{
public int Value { get; set; }
public static bool operator ==(specialInt op1, string op2)
{
return true;
}
public static bool operator !=(specialInt op1, string op2)
{
return false;
}
public override int GetHashCode()
{
return Value;
}
public override bool Equals(object obj)
{
return true;
}
}
have we guaranteed bug free software?
Wow - what can I say... what a great time!
Things I get to cross off my todo list:
* Rock a 500+ note streak on expert in Rock Band in front of 200 or so people in the Hendrix museum
* Meet most of the F# team
* Ask Ray Ozzie of Groove is going to be the new front end to MOSS
* Hang out with a bunch of really cool friends, such as Bock, Chris Williams, St. Jean, Rocky, Keith Franklin, and numerous other people who were previously just bloggers on the other end of the terminal until this week
* See the future of C# (and no, will not be blogging about it... darn NDA!)
* Talk about the merits of agile, doing agile fixed bid projects, and software quality in general, with Jeff Palermo among other really cool people from the "are cheap developers expensive" open-spaces session
At least that is what I can remember at the moment. All in all, a really, really great time. Very much looking forward to TechEd now where I should get a chance to see some of these same folks again. Thanks to the team at Microsoft for putting together such a great event!
I was looking for where the § is on my keyboard, as many of the Linq to SQL samples in F# use that in front of the database context variable.
Strange enough, was looking on my keyboard for it, with no luck. The key does not exist, at least on my laptop. Thus, I see a market opportunity.. an add-in for vs, possibly on codeplex, that puts § as a toolbar button so those of us with US keyboards will still be in luck :) Of course, I wonder who DOES have that on the keyboard.
Truth be told, when I read that in the Expert F# book at first, I thought it was a typo... apparently, no!
Lets hope that part of the spec changes... soon :)
Wow - just realized that our feed now has some great content from Magenicons that do not use this particular community server instance... aka, the Magenic Mirrors blog. Corey Miller, Dan English, Aaron Lowe, are all there, with many more being added. Check them out... lots of great stuff. Aaron Lowe is an wicked smart SQL Server guy, Corey Miller is a Silverlight Stud, and Dan English is a BI Ace. Definitely check it out.
Yes, I gave in and created a Twitter thing. URL:
http://twitter.com/AaronErickson
InfoQ has spelled out what is coming with the next major release of F#.
A lot of this stuff will be very useful. Having interactive F# inside of VS will be nice just to not have to switch apps to move between interactive and "coding" mode. As well, it looks like we will get improved intellisense, which for someone like me who is still learning the language and stumbling through the API, will be a godsend.
What I really hope though is that the compiler steps up the descriptions the compiler errors. The syntax of F# is very different than what a lot of people who aren't already "functional geeks" are used to, so knowing what your error is beyond the ubiquitous "syntax error" you get when you highlight your cursor over the red squigglies in VS will be very helpful to people getting up to speed.
Again, always a huge pleasure to attend one of the most fun events that I do, which of course, is the Twin Cities Code Camp that Jason Bock puts on twice a year.
What a fun event. The people up there are great - everyone is really curious, and the audience participation is always fantastic. I had a ton of fun getting to talk about F# to some very interesting folk who I hope got anywhere close to as much out of it as I did learning about the whys and whens they are interested in such a forward looking technology. Hope to do it again soon...
Normally, I don't post here about "personal" projects, but since I needed a good reason to learn F#, and I am somewhat of a political junkie, I have launched scandalgenerator.com, a website that all the campaigns can use to streamline their operations.
The site is designed to make it possible for any of the major presidential political campaigns to generate scandalous headlines. If the first result isn't to your liking, just hit refresh until you find a suitable headline, and viola, all done!
For the super curious, the source code that generates the headlines is here.
Official announcement via dotnetrocks.
Download it here.
This kind of thing is the favorite part of my job, bar none.
We are approaching release for CSLA 3.5, which includes what Rocky and I are calling "Linq to CSLA".
Of course, there is nothing much new about this - I have talked about a lot of these same ideas before. That said, it is notable to point out that Linq to CSLA, which is implemented as a custom IQueryable, allows you to query across CSLA collections (BusinessListBase and ReadOnlyListBase derived objects), and get in return objects that you can do bi-directional binding with. This is, of course, opposed to the normal results of a Linq to Objects query, which will typically result in a sequence which, while allowing for bi-directional binding on the items within the collection, does not result in collection level changes making it back (i.e. add an item to a grid bound to a Linq to objects resultset).
The details around this are that when you do a query that uses an "identity projection" - that is, project the same object you are querying for, rather than getting back a sequence, you get back a LinqBindingList which reflects collection level changes back to the collection that you performed the query on.
In addition to this, which frankly, was a key to make Linq work with CSLA as most would expect it to, we went ahead and added i4o style indexing to CSLA as well, so you can use the [Indexable] attribute to your child classes and have CSLA index those both within BusinessListBase and ReadOnlyListBase.
Speaking of i4o, I do have a refresh of that project coming later in April, once I get back from MVP summit... some important performance updates and functionality coming to that as well, including much faster index creation that depends less on reflection that some readers have submitted... so stay tuned.
In the meantime, feel free to head to Rocky's website and download the beta of the new CSLA - feedback is certainly appreciated.
Not content to remain silent this political season, as a service to all the campaigns, I am providing some source-code for a "scandal generator" - an automated tool the candidates can use to generate headlines and save needed campaign donation funds.
Of course, this could be put behind a website, an automailer to press organizations, or used in a number of other clever ways. Basically, you feed it a file that represents a madlib (line 1 is the headline, the rest of the file is the body) - and it produces a "scandal" object that should have everything you need.
Oh, and the cool part, it is written in F# :)
Have fun!
#light
module AaronErickson.ScandalGeneratorEngine
open System
open System.IO
let private probabilityTest (range:int,prob:int) =
Random().Next(range) < prob
(* various named probabilities *)
let private isImportant() = probabilityTest (100,50)
(* our politicians *)
type Politician = { Name: string; FullName: string; ProNoun: string; HomeTown: string}
(* opposing politicians *)
type OpposingPoliticians = { Politician: Politician; Opponent: Politician }
(* story type *)
type Scandal = { Headline: string; Body: string; IsImportant: bool }
let private obama = { Name="Obama"; FullName="Barack Obama"; ProNoun="his"; HomeTown="Chicago" }
let private clinton = { Name="Clinton"; FullName="Hillary Clinton"; ProNoun="her"; HomeTown="New York" }
let private mccain = { Name="McCain"; FullName="John McCain"; ProNoun="his"; HomeTown="Phoenix" }
let private getPoliticians() =
match Random().Next(3) with
| 0 -> { Politician=obama; Opponent=clinton }
| 1 -> { Politician=clinton; Opponent=obama }
| 2 -> { Politician=mccain; Opponent=obama }
| _ -> { Politician=mccain; Opponent=clinton }
let private getPressAgency() =
match Random().Next(3) with
| 0 -> "The Onion"
| 1 -> "The Drudge Report"
| 2 -> "Fox News"
| _ -> "The Associated Press"
(* generates a single scandal composed of a headline, a story, and whether it is considered 'important' *)
let GenerateScandal (templateFile:String) =
use reader = new StreamReader(templateFile)
let templateTextHeadline = reader.ReadLine()
let templateText = reader.ReadToEnd()
let politicians = getPoliticians()
let headlineWithPoliticians = templateTextHeadline.Replace("%politician%",politicians.Politician.Name)
let storyWithPoliticians = templateText.Replace("%politician%",politicians.Politician.Name)
.Replace("%fullpolitician%",politicians.Politician.FullName)
.Replace("%opposingpolitician%",politicians.Opponent.Name)
.Replace("%pronoun%",politicians.Politician.ProNoun)
.Replace("%hometown%",politicians.Politician.HomeTown)
.Replace("%pressagency%",getPressAgency())
{ Headline = headlineWithPoliticians; Body = storyWithPoliticians; IsImportant = isImportant() }
.. is not quite as easy as it should be.
In LINQ to SQL, when you execute a query, it seems like dependent properties (i.e. properties that are objects formed based on relationships between tables, think Customer-->Order)... are lazy loaded by default.
Now, I do believe that is probably best - since in general, when you do a query over one table, it will be hard to know in a complex graph which tables to eager-load and which tables and rows to lazy load.
The solution is to use the DataLoadOptions class to specify which dependent properties you want to "eager-load".
So, say, you are doing the following query:
//sets up eager loading on most of the dependent properties
data.EagerLoad();
//grab the raw data we want... will group later
var oddsDataRaw = from odds in data.OddDTOs
where odds.GameDTO.GameDate >= startDate
&& odds.GameDTO.GameDate <= endDate
&& odds.GameDTO.HomeTeam.TeamSeasons.Seasons.Leagues.Initials == sport
select odds;
//convert the data into a list, then group it
var oddsData = (from o in (oddsDataRaw.ToArray())
group o by o.GameID).ToArray();
Of course, you might wonder what the EagerLoad routine does... :). Well, yes, there is no EagerLoad() - I wrote it as an extension method:
public static void EagerLoad(this WTDataContext dataContext)
{
DataLoadOptions options = new DataLoadOptions();
options.LoadWith<OddDTO>(o => o.GameDTO);
options.LoadWith<OddDTO>(o => o.OddsMakers);
options.LoadWith<GameDTO>(g => g.AwayTeam);
options.LoadWith<GameDTO>(g => g.HomeTeam);
options.LoadWith<Team>(t => t.TeamSeasons);
options.LoadWith<TeamSeason>(ts => ts.Seasons);
options.LoadWith<Season>(s => s.Leagues);
dataContext.LoadOptions = options;
}
Now, prior to doing this, reading stuff from my object graph that resulted from the initial query took a very long time. I am consolidating data here from about 6 normalized tables. Prior to setting up data options, the query and resulting mapping to my business objects took about 30 seconds. Given this runs behind a web page that summarizes stats for sports games, that isn't really going to be viable.
Calling the EagerLoad extension method allows the query to load up the related objects from the graph according to the specification that gets built with the LoadWith routine. Specifically, you call LoadWith by passing the parent object type as the generic parameter, then passing a Lambda that has the property of the said type to "eager load". In fact, LoadWith could have been called "EagerLoadThisClass" instead, and probably conveyed more meaning (but I digress).
Of course, it would be nice if there were a way to have a syntax that went like this:
dataContext.EagerLoad(Delegate someCode)
which would, of course, set up your options based on the content of the delegate (or some expression set representing the delegate).. but I will leave that as an "exercise to the reader" :)