There is a lot to learn from Fabien Potencier, the creator of the symfony framework. He often comments people's work on other frameworks, but almost never when someone works on his own framework. So his recent reaction about my DDD experiment is rare enough to be thoroughly analyzed, and distilled. Let's look for the very substance of his latest post.

The Good

It's been more than ten months since my original Forms post, about which Fabien basically told me that I knew nothing about programming (which is true) but nothing else. So I'm very glad that he finally decided to give more feedback about the ideas I suggest. My previous post asked for developers' thoughts about a reworked Chapter 10, and who could better do this than him?

Not only does Fabien talk about my DDD experiment, he actually implemented some features I suggested. Thanks to commits made to the symfony 1.2 branch early this week, building a form object using the sfForm class alone is easier. Added is the ability to define default values for each widget, the ability to iterate on a form object, the ability to directly set a widget or a validator from the from object. I proposed all those changes to make the documentation easier to read (and write), and it seems that they also have an interest for the developers.

Fabien points some mistakes I did, like the 'multiple' validator, which is a bad idea. Since there are two kinds of multiple validators (on the uncleaned value and on the cleaned value), the 'multiple' keyword is not the best choice. I still thing that 'pre' and 'post' validators could get a better name, but that's a detail.

Also, the modified version of my proposed Chapter 10, published as an attachment to Fabien's post, keeps about 90% of the original text unchanged. It seems that he disagrees mostly on some technical details, and didn't touch the order in which things were introduced, nor the length of the Chapter.

The Bad

But the idea to define form widgets and validators based on an associative array got no credit to his eyes. It is probably a matter of preference; I introduced this array syntax for two reasons:

• To avoid introducing too many classes too early in the documentation
• To make the YAML form definition syntax completely natural

To my eyes, this array syntax has always been a layer on top of the existing syntax; that means that the ability to define custom widgets and validators is still there, and the ability to pass an object instead of an associative array is still there as well. That's how I tried to remove the coding style preference problem: whatever you like more, symfony supports it. You get both simplicity of use and power.

"[The suggested API] is not shorter, it is not easier to understand, and it is more difficult to explain.". Let me respectfully disagree. It is shorter, easier to understand, and easier to explain. Compare what a single chapter explains and the confusion introduced by an unfinished and lengthy book. Or better, compare:

And:

# in YAML
&subjects:     [Subject A, Subject B, Subject C]
name_format:  contact[%s]
id_format:    my_form_%s
widgets:
  name:       text
  email:      { type: text, default: me@example.com }
  subject:    { type: select, choices: *subjects }
  message:    textarea
  file:       file
validators:
  name:       { type: string, required: false }
  email:      email
  subject:    { type: choice, choices: *subjects }
  message:    { type: string, min_length: 4, errors: { min_length: Your message is too short } }
  file:       file

According to Fabien, using associative arrays to make the YAML syntax easier to explain is of no use, since YAML is bad and shall be dropped altogether. "Learn from our mistakes" means that symfony should never had used YAML in the first place, despite the fact that it appealed numerous users to it and that it's only a simplicity layer. That means that the symfony 1.2 admin generator will not be controlled from a YAML file at all - defining form widgets in YAML is an indispensable brick to a YAML syntax for an administration interface. So be prepared to use XML or plain PHP for your database schemas, configuration files, generated modules, etc.

"Learn from our mistakes" also means not using strings to define HTML attributes anymore. I suggested to keep on using the abilities of symfony 1.0 to output clean XHTML attributes from a string looking like 'id=contact_subject class=bar'. But that is something else you should forget about. Apparently, this brings no benefit over array('id' => 'contact_subject', 'class' => 'bar'). Once again, I can understand it's a matter of preference, but I'm convinced that this kind of syntactic sugar is what appealed many users to symfony in the first place.

In the documentation I wrote, I introduced a way to bind a form object to the request object ($form->bind($request)). I explained later in the chapter why it's better to do otherwise, but at least it shows that a form can be used without necessarily using the array syntax for widget names. Fabien explains why this is wrong (as I did) and fails to see the interest of introducing the setNameFormat() in the documentation only when it becomes necessary. However, the current symfony book uses this technique several times (think of in Chapter 2 for instance), because it is easier to know why a practice is wrong once you've seen the advantages of the good practice in comparison. His revised version of the Chapter 10 doesn't solve the problem, either.

But honestly, I don't care much about all these points. If if was just for Fabien's remarks on the technical side of things, I'd be more than willing to continue working on a modified Chapter 10 to make it worthy of The Guide.

The Ugly

But Fabien is really going to a nasty place with his post.

Does he bring a response to my request for comment? No, he replies to a commenter of my post, who challenged him to give his opinion. He never actually addresses me directly - I'm a persona non grata.

Does he agree on the DDD experiment? Of course not, DDD is the "Biggest problem" according to him (not sure why). How could documentation and teaching influence a developer's design decisions? The forms API is "good enough" as is, and its lack of documentation is not a problem to be solved by modifying the code. If someone (but not me) wants to write something to "help us improve the current documentation", he can still try again.

Is he grateful that I did in a week a work (Chapter 10) that he couldn't do in a year? Not at all. Not a word of thanks, he just feels insulted that someone dared to question parts of his work. What a childish reaction to someone who offers to help widen the symfony adoption.

Does he react in the interest of the community, proposing to use his own version of the Chapter 10 for the current guide? Not even that. He prefers no documentation at all rather than an equivalent to the symfony 1.0 documentation updated for symfony 1.1.

Does he care about the newcomers to symfony, those who don't know the 1.0 API by heart, don't follow the Trac timeline every day, don't read the framework code, and don't accept an UPGRADE text file for a documentation? No. Not a word about them in his post. Only the developers who already know good practices of web development can start using symfony. You can no longer learn these practices by learning symfony.

Does he write that he's been implementing some of my ideas? No, that would be giving me too much credit. The goal, here, is to show that I am a bad developer, and nothing else. Well, I'm not even a developer, so why all the hate?

Is he trying to be constructive? No. He writes, in bad faith: "[Francois'] API is so unintuitive that we must explain a lot of things to describe the way it works". God, I thought that I managed to explain in 1 hour what he needs a day to teach in an expensive training, and it's my API that is unintuitive?

Conclusion

All in all, Fabien reacts with pride rather than reason. He does as much as he can do discredit my work, while my purpose has always been to help leverage the symfony adoption. He probably dreams that, with a single blog post, I'd leave the symfony community completely, because he finally demonstrated that none of my work is worthy to his eyes.

Too bad, Fabien, you have taken the wrong path. I'll be a pain in your ass for a long time. Count me in to constantly remind people that symfony is a one-man work, and that this is a very high risk for enterprise projects, given the man.

If you've watched or read my presentation on Documentation-Driven Development, you may wonder how to put that new methodology into action. A practical example is often better than a long explanation, so let's see ho to apply it to the new Forms sub-framework introduced by symfony 1.1.

Not DDD

In order to use the new sfForm library, you must either read a book (not yet completely written) or dive into the source code and guess how to use it. To my mind, this is pretty much the contrary of what leads to a large adoption.

The Form framework was designed with power in mind, and reaches this goal very well: you can use it to create forms of any level of complexity, including forms embedding other forms, forms with a variable number of fields, forms split into several steps ("wizards"), etc. It is very much object oriented, so everything can be reused or overridden.

But unfortunately, in order to create a simple form, you need to learn a lot more and write a lot more code than what you used to do in symfony 1.1. The current Forms documentation describes the API and justifies its implementation. It goes very much into the details of each part of the sub-framework, and quite early in the learning process. The result - for me, at least - is that the reader feels overwhelmed by the huge amount of classes, features and options, and dismisses the whole sub-framework for being too complex.

"Let's use that new Form stuff for complex forms and keep the current form helpers and YAML validation for everyday forms", I hear. That's a pity, because once you understand how the new Forms sub-framework works and accept its verbosity, there is no good reason to stick with the old system.

An Ideal sfForm Documentation

I think that a piece of documentation is missing. This piece is probably an introduction to the Form sub-framework.

In symfony 1.0, a single chapter of the book was enough to master forms for most use cases. Even if the new form sub-framework is more powerful than the 1.0 one, it should not be more complicated to learn and use in similar cases. So the sfForms introduction should be short, requiring at most one hour to read it.

After reading this documentation, an average developer should be able to use sfForms in 80% of the cases. That includes at least all the features described in the original Forms chapter of the symfony book:

• Displaying a form
• Available form helpers
• Displaying a model-based form
• Dealing with Foreign keys
• Handling a form submission
• Validating a form
• Available validators
• Repopulating a form
• Complex use cases

The target audience would be people knowing some concepts about symfony, but not yet everything. In fact, they should know what the Chapters 1 to 9 of the symfony guide cover, not more. So some advanced concepts should probably be skipped, or explained only after the fundamental usage is clear.

This introduction should not require additional lookup in the Forms book. That means that it should be self-sufficient. It probably also means not including the justifications of the Forms implementation that you can find in the current Forms book. The reasons why the API was designed the way it is should become obvious at the end of the introduction. Expert customization and rare use cases should probably also be left aside.

The symfony 1.0 documentation introduces concepts and features in a certain order, with a precise purpose: not loading too much information into the reader's mind at a time. In a similar fashion, the forms introduction should be a linear piece of documentation, not a set of articles that you can read in any order with hyperlinks everywhere to break the reading flow.

The forms framework is powerful, but the current form book somehow translates that into length, and verbosity. On the contrary, I think the reader should feel exalted: the documentation should put him in a rush to start using the new forms. So the forms introduction should "tell a story", and gently lead the reader to a point where he feels he can grab the steering wheel and drive the car by himself.

API enhancements

The problem is that explaining the current API takes much longer than a single piece of documentation. That's because of the many options available, because of the many objects to learn, and because even the simplest things (like a list of form controls) look complicated (sfWidgetFormSchema).

There is not much choice to overcome this problem. In order to write a short and readable guide to the forms sub-framework, its API must be adapted. That's right, the API must be changed so that the documentation can be made shorter, and more usable. This is one of the principles of the Documentation-Driven Development methodology.

These API enhancements should be completely backward compatible, so that any existing application using the current sfForms implementation can continue to work seamlessly with the modified implementation. In a way, that qualifies the API enhancements as a simplicity layer on top if the existing code. As a side note, the current Forms book still remains indispensable for advanced usage.

Note that the API enhancements don't need to be implemented before the new documentation is published. The implementation comes second, after the documentation. That's another of the DDD principles: explain first, make it work afterwards. After all, project managers write requirements for web applications before they exist, all the time.

Do As I Do, Not As I Say

Some people are getting sick of reading me criticizing parts of the symfony framework. Well, I'm not criticizing: I'm actively improving.

Rethinking sfForms is a good example for a Documentation-Driven Development. To illustrate this methodology, I'm going to rewrite the Chapter 10 of the symfony book for symfony 1.1. That's right, the current Chapter 10, which describes the "old way" of doing forms, can be rewritten in a similar fashion and serve for symfony 1.1.

But since the current API requires too much explanation to be used, I'm going to introduce the necessary API changes to the sfForms library. I'll create and manage forms in a way slightly different from what the current API allows, to make it simpler to use - and to explain.

When the new Chapter 10 is published here in this very blog, this piece of documentation will be of no use since the features it describes won't be implemented yet. But I know that writing documentation is not enough to convince people (yet), so I will Implement the API changes as a second step to the exercise. As I'm not a very good developer, any help will be welcome during that phase (contact me If you want to give me a hand after the documentation is published).

If everything goes well, the implementation of the API changes will be be released as a symfony plugin - maybe called sfSimpleForms. I hope it can lead more developers to adopt the greatest open-source Forms framework around.

Did you attend this year's Symfony Camp? It was a great event, the unique occasion to meet the core team of the symfony framework. I had the opportunity to give a talk there, and you can now watch it online:

Don't hesitate to comment on this presentation on SlideShare.

Thanks to all the great people that I met there who gave me a feedback on my work, encouragements or advice. Thanks to Dutch Open Projects for the organization - and for inviting me. It was a great pleasure to exchange about symfony, its past, present, and future, with so many enthusiastic people.

Update: It seems that my slideshow has been featured on the SlideShare homepage by the SlideShare editorial team.

I recently installed the PicLens Firefox extension. It is an incredibly useful way to browse image collections, the interface is both very responsive and well thought, and the integration into existing websites is unobtrusive enough to convince me.

Then, as I was monitoring requests on one application I develop on my local server, I noticed that each time I requested a page, two requests were received by the web server (in addition to requests for web assets such as JavaScript, CSS and image files). After investigation, I realized that the PicLens extension detected a <link> tag in the page content, and automatically fetched the RSS feed linked by that tag. It does so everytime it detects an application/rss+xml link.

I made the test with pages including more than one RSS feed (try php.net for instance) and noticed the same behavior, only at a larger scale. So PicLens does basically what Google Web Accelerator does: it prefetches web resources (in this case: RSS feeds) to accelerate the navigation experience.

I emailed the PicLens support about the issue, and here is their response:

Hi Francois,

Thank you so much for your kind words and for using PicLens! We really appreciate you taking the time to send us your thoughts.

I'm sorry to hear you are worried about PicLens's prefetching behavior. We prefetch all tags that have a content type of "application/rss+xml" because we use that to match up mediarss feeds with items on the page. It's not a bug at all, nor have we heard of it causing any problems for anyone. Is there a specific reason you feel that it jeopardizes websites?

Hope to hear back from you soon.

All the best, Meg & The PicLens Team

I can think of many reasons why link prefetching is bad, among which wrong statistics, additional bandwidth and server load. But maybe I'm being too extremist on that one. What do you think? Can prefetching be considered as an acceptable practice nowadays? Or is the PicLens extension something that should not be installed?

I'm officially retiring from the symfony project core team and documentation. My recent rant about the shift in symfony philosophy lead Fabien and I to agree that we disagree. The symfony project will have to find new documentation writers.

I've been working on the symfony project for three years now, and it's been a great adventure. Seeing what was originally a very specialized piece of work done by the head of a French Web Agency become one of the top web application frameworks in the whole world has just been amazing. Betting on Open-Source, dedicating time and money to develop and document an internal tool was a great move for Sensio. I learned a lot, thanks to Fabien and Sensio. I enjoyed writing the Askeet tutorial, the symfony book and all the other tutorials very much. I enjoyed writing plug-ins and applications, just for the pleasure to donate them. Now that Big Players recommend symfony for large-scale applications, a bright future opens for all the symfony developers out there.

I've also been battling for three years. I kept trying to put my two cents in the symfony development choices, even though I'm not a developer. I kept advocating simplicity of use over sophistication and code purity. I kept urging for smaller, more often releases. I kept giving the documentation writer's point of view, which is that if something is hard to explain, it is probably also hard to use. I kept asking for more methods, so that there could be More Than One Way To Do Things - one simple, and one powerful. To a certain point, some of my remarks have been taken into consideration, but it required a tremendous amount of energy and stubbornness. And even now, I'm reminded that these changes (like using YAML instead of XML for schemas for instance) were mistakes.

Today, my opinions are clearly not the ones of the "symfony core team". Important design choices are not discussed with the community, just like when symfony was only developed internally. 95% of the code base is still the result of a single man's work and decisions. The community is just there to provide support, do the beta testing and play with plugins. This is a big strength of the project (no dispersion, no time lost in sterile discussions), but it's also frustrating when somebody wants to get more involved.

I left Sensio last year partly because I couldn't get my views accepted. I'm leaving symfony today for the same reason. I regret it, because I would love to continue collaborating to the project, because I really like writing documentation, and because I love the open-source spirit and the symfony community. But the time I spend on symfony is now on my free time, and I can't just do what others decide. A hobby must be fun, not frustrating. I'm tired. Three years is long enough.

I also regret to leave a half-finished symfony 1.1 guidebook. But I heard that Sensio is writing a whole book about the new Form system (!), and I'm confident that they can dedicate internal resources to writing documentation. If you're interested in contributing, you should get in touch with Fabien - he's now the documentation lead (in addition to the rest).

To conclude, I'd like to quote a French guy named Talleyrand. He said: "War is much too serious a thing to be left to military men". I personally think that Programmatic API is much too serious a thing to be left to developers.

Among the remarks I have about symfony 1.1, the most recurring one is the shift of philosophy between the 1.0 and 1.1 syntax. If symfony 1.0 syntax was made to write code fast, I believe it is not the case anymore with symfony 1.1, which is designed primarily for extensibility.

The result is that a symfony 1.1 application looks a lot more like a Java program. I tend to agree that Object-Orientation is a good thing because it forces you to organize your code in a modular way. But when object-orientation makes you need to keep a symfony book aside at all times and multiply the number of LOC by two, I think it's a dead end.

Disclaimer: I'm not a developer by training. I don't have much experience in web development. I started learning PHP about three years ago, and I don't think that code purity is something you should stick to. I believe in code smell, though, but that's another matter. My main interest is usability in general.

Example #1 - Tasks

Let's see how the new symfony philosophy applies to the new command line utility framework: the symfony tasks. This is a very powerful addition to symfony 1.1, made to allow an easy parsing of CLI arguments and options, and to write CLI scripts in an extensible way.

Here is a typical task initialization method, taken from sfLogRotateTask (for details about the syntax, please refer to the symfony 1.1 documentation):

To be able to write a task by yourself, you need to know what an sfCommandArgument is, what an sfCommandOption is, and you need to know their constants as well. That means that the number of things to learn is quite important. And why do we need to instantiate a new object to set arguments, since the only thing that addArguments() accepts is an array of sfCommandArgument objects? Because, sometime, you will take advantage of the fact that you can pass a subclass of sfCommandArgument to extend the task framework.

Developed according to the symfony 1.0 style, this would probably give:

Do you see the difference? The latter is maybe not as easy to extend, but for 90% of the cases, faster to write. Besides, the addArgument() method does not prevent you from using the long version with addArguments(), so you get both extensibility and brevity.

Note: By reading the above code, you don't know what the null and true parameters stand for. It is a matter of taste, and longer to write, but I'd rather have my method calls look like the following when the method takes many arguments. That way, every method call in your application is a useful reminder of the API.

Fabien added an addArgument() method to sfTask not so long ago, but it just seemed useless to him to duplicate the method signature. I think he did that just to make me stop yelling. You, developer, should learn The Right Way of Doing Things, and the right way is the longer way. If you can't understand or learn the longer way, bummer, you can't use symfony.

Example #2 - Events

The event system is another great addition to symfony 1.1. It extends the principle of mixin to allow more runtime class modifications. Once again, the symfony book is up to date on this part, so you should read the related chapter if you need to understand the syntax.

So here is how to allow a class to be extensible by way of a generic __call() in symfony 1.1:

And here is how you register a new bar() method with this system:

As I wrote the documentation for this feature, I couldn't refrain from thinking this could be a lot easier. With symfony 1.0 philosophy, I guess you would only need to write:

And you would probably register a new method this way:

You can guess how the handle() and addMethod() methods of sfEventDispatcher could do all the dirty job for you. That would save you a copy/paste from the first version of the code, would be less error prone, and after all, if the program has less lines, then it's cheaper to maintain.

But, according to Fabien, that would be hardcoding the way you throw an exception in __call(), and this addition would only be useful in a very minority of cases. While the fact that you pass to notify() an sfEvent instance instead of a simple list of parameters is, once again, based on the idea that you will sometime find it convenient to subclass sfEvent. Which I think will never happen before symfony 1.2 at least.

Example #3 - Forms

Don't get me started on forms. I already wrote a post about sfForms to say that it missed an important part, and that code usability had been left aside. I still think so. Fabien doesn't.

Oh, by the way, do you know that the First Project Tutorial in symfony 1.1 showcases the new Form framework? If that doesn't discourage newcomers, I think nothing will. I wish good luck to the one who will try to make this tutorial newbie-friendly.

Conclusion

Symfony 1.1 adds a ton of wonderful features to symfony 1.0. From a technical point of view, this is a more complete, more coherent, more beautiful piece of code than symfony 1.0 was. It kicks ass and I believe that it's more powerful than any other PHP framework out there.

On the other hand, I think that symfony 1.1 is more verbose, and therefore more error-prone, than symfony 1.0. It forces you to constantly look at the book for syntax description. The book cannot describe everything - it is just a guide on how to use the framework. It should remain as small as possible to keep readable. The book cannot go into the details of sfCommandArgument::REQUIRED or sfWidgetFormSchema. That means the book will not be enough to use symfony 1.1: You will need to know the full API doc as well, even for the basic uses. Fabien knows the API by heart, so he doesn't need any documentation. But you, my friend, you will sweat a lot to learn all the changes.

The heart of the problem is that, to use symfony 1.1, you don't just need to learn the usage guidelines, you also need to understand how it works inside.

I disagree with the philosophy shift symfony is taking. I think it should stick with the "KISS" motto, and it could do so without sacrificing extensibility. If the price to pay is a little more code in the symfony base, then symfony should pay this price - not the symfony users.

I'm tired of writing a book that looks more and more like an API documentation. I don't feel like explaining the core symfony classes to developers who just wants to use the framework to build an application. I think that The Definitve Guide To Symfony doesn't really make any sense anymore. You want to know how to use the framework? Well, read the API doc and guess. After all, that's what all the other open-source projects do.

Maybe I'm the only one thinking that. But, as I said at the beginning, I'm no developer.

Doing user testing is more or less like watching users interact with an application. sfSpyPlugin allows you do to it online and for free, instead of paying a lot for doing it for real.

User testing is usually a long and expensive process that few organizations can afford. You must recruit users, write scenarios, animate the tests, record browsing sessions, and visualize the tapes. What if you could just choose some of the users currently using the application online, start recording their actions, and playback their way through the application later? That's what the new sfSpyPlugin offers.

See it in action: One browser window is "spying" another browser window - but that could very well be two different computers. Click on the following link to see a 2 minutes screencast of the plugin in action.

http://www.screencast.com/t/nLi4Vrq9ND

The plugin displays a list of the online users on any symfony application. You can choose to watch online or record for later the actions of any user. The replay interface offers VCR-like controls to pause, restart, accelerate or slow down the playback. You can also organize your recordings by giving them a name. The interface should be pretty self-explanatory, and yet there is a complete installation and usage guide available at the symfony project wiki.

And if you are not of the point-and-click type, you can always trigger the recording from code, to watch how a certain user uses the application, or how a certain module of the application is used.

This way, you can watch and monitor how users interact with your application, or part of it. The only think you will miss is the voice of the user commenting what he/she is doing. I created this plugin so that you have no excuse to let live applications having a poor usability without doing anything to improve them...