Abandoning hope... and XForms
Sometimes a decision must be made. I have invested many hours in a standard that is quite complex and tedious to maintain, and that was great fun, but enough is enough.
Why did I invest in XForms when it took so much effort and time? Because I believe standards are good and good standards come from the W3C and the XML community. Also, I always had somewhat of an inferiority complex in IT development that I tried to compensate by using techniques supposedly invented for common folk, i.e. non-programmers. Both are bad reasons from where I now stand. I took up XForms because I was infected by the enthusiasm of a friend (who was even quicker to abandon it) about 3 years ago. At first I assumed it just used a model from which to infer a form. It turned out it was like a superset of HTML forms with options for displaying and validating data. This was largely taken up by HTML5, as we all know. The Model remains the core of XForms, but the W3C is slow to respond to innovations and the need for a better user experience, while the programming community is evolving rapidly and growing steadily. The needs and expectations for dumbed-down tools is shifting accordingly.
When it comes to the model of XForms, there is a gap with HTML5. The model separates the form elements from their respective types, constraints and representations. But wait, is there really a need to separate these? Why not have all types, constraints and representations in the document proper? Ah, because of re-usability. Well, to be honest, so far I never encountered a use case where I could actually reuse a model! It never mattered if I declared binds on the elements or in the model, and I can't quickly think of a use case where it should.
Back to my first intuition of generating a form from a model. I must have been quite stupid back then, because I was obviously thinking about a Schema... As it turned out, people have problems grasping the difference between a model and schema, and to be honest, so do I. If a model is not generic in any way, than that means it is just another document. I recently found out that this is the core problem that I encounter when working with XForms. Like HTML5, it is document-centric, not data-centric. That was my error.
The first time I thought of abandoning XForms started with a technical debate on the usage of a specific JavaScript toolkit on the betterFORM users mailing list. I was brooding on something all this while, but couldn't grasp the issue. Now I get it: the problem was not the choice of client toolkit for the job, but the problems that arise when implementing a document-centric solution in a data-centric environment. JavaScript has taken a leap forward since toolkits like Dojo opened the possibility to switch between declarative (document-centric) widgets to programmatic (data-centric) widgets. Since then a lot of patterns have emerged that deal with problems that arise when binding events and methods in a document. To be able to benefit from these solutions, you will have to do it the Dojo way. Clearly, this my-way-or-the-highway approach is not particularly friendly towards other document-centric solutions. HTML5 will allow for a standard to be developed in tandem with dojo/event and dojo/method, XForms probably won't.
At some point I made a decision to only use programmatic widgets. The power of JavaScript engines these days is enormous, and smart design allows for a seamless user experience. At this point the incommensurability with XForms became most apparent. My use case here is localization: a website that needs to have forms in two languages.
At first my idea was document-based: use a different form for each language. Maintenance was of course killing, but the thought was correct. Much later I decided to translate the forms after all. How to go about it? I read something about putting a static key/value map into the model. Bad idea, right? I also thought about other solutions. I could, for instance, add selectors to my form elements in XForms and map those in the client from a different, more flexible location. Or I could translate the form using XSLT or Xquery, and apply xforms to an already translated document. As per betterFORM documentation I chose the first, and it's a mess. How does Dojo solve this? Each widget has a language attribute and can be assigned an nls object from its own namespace or anywhere else. The object is of course malleable at runtime, and if it's not available a default language is selected. As an aside, betterFORM does use the Dojo locale, but, alas, incorrectly (in 4.1).
There is no way I can have the same power in a declarative way when it comes to localization. As it turns out, the publishing company I work for published a standard work on localization. Perhaps I should read it to find my assumptions are wrong. But apart from localization I think the issue remains. Forms are widgets, rarely used in inline text. They should be approached data-centric, at least on the client. When form data needs to be validated or stored on the server it should be done in parallel with processing on the client, which in my opinion should be leading (I'm staying far away from HTML5 forms too). Forms can be easily generated from simple types and a schema. Creating a schema in a GUI is much more user-friendly than writing XML, so this is a MUST. Even if that GUI would create XForms, forcing its rules on a user is a no-go. Yes, I have ideas for such a GUI, but they are not part of this article. The topic was abandoning hope and XForms. Perhaps now it is only fair to admit that I never had much hope for XForms anyway...
Why did I invest in XForms when it took so much effort and time? Because I believe standards are good and good standards come from the W3C and the XML community. Also, I always had somewhat of an inferiority complex in IT development that I tried to compensate by using techniques supposedly invented for common folk, i.e. non-programmers. Both are bad reasons from where I now stand. I took up XForms because I was infected by the enthusiasm of a friend (who was even quicker to abandon it) about 3 years ago. At first I assumed it just used a model from which to infer a form. It turned out it was like a superset of HTML forms with options for displaying and validating data. This was largely taken up by HTML5, as we all know. The Model remains the core of XForms, but the W3C is slow to respond to innovations and the need for a better user experience, while the programming community is evolving rapidly and growing steadily. The needs and expectations for dumbed-down tools is shifting accordingly.
When it comes to the model of XForms, there is a gap with HTML5. The model separates the form elements from their respective types, constraints and representations. But wait, is there really a need to separate these? Why not have all types, constraints and representations in the document proper? Ah, because of re-usability. Well, to be honest, so far I never encountered a use case where I could actually reuse a model! It never mattered if I declared binds on the elements or in the model, and I can't quickly think of a use case where it should.
Back to my first intuition of generating a form from a model. I must have been quite stupid back then, because I was obviously thinking about a Schema... As it turned out, people have problems grasping the difference between a model and schema, and to be honest, so do I. If a model is not generic in any way, than that means it is just another document. I recently found out that this is the core problem that I encounter when working with XForms. Like HTML5, it is document-centric, not data-centric. That was my error.
The first time I thought of abandoning XForms started with a technical debate on the usage of a specific JavaScript toolkit on the betterFORM users mailing list. I was brooding on something all this while, but couldn't grasp the issue. Now I get it: the problem was not the choice of client toolkit for the job, but the problems that arise when implementing a document-centric solution in a data-centric environment. JavaScript has taken a leap forward since toolkits like Dojo opened the possibility to switch between declarative (document-centric) widgets to programmatic (data-centric) widgets. Since then a lot of patterns have emerged that deal with problems that arise when binding events and methods in a document. To be able to benefit from these solutions, you will have to do it the Dojo way. Clearly, this my-way-or-the-highway approach is not particularly friendly towards other document-centric solutions. HTML5 will allow for a standard to be developed in tandem with dojo/event and dojo/method, XForms probably won't.
At some point I made a decision to only use programmatic widgets. The power of JavaScript engines these days is enormous, and smart design allows for a seamless user experience. At this point the incommensurability with XForms became most apparent. My use case here is localization: a website that needs to have forms in two languages.
At first my idea was document-based: use a different form for each language. Maintenance was of course killing, but the thought was correct. Much later I decided to translate the forms after all. How to go about it? I read something about putting a static key/value map into the model. Bad idea, right? I also thought about other solutions. I could, for instance, add selectors to my form elements in XForms and map those in the client from a different, more flexible location. Or I could translate the form using XSLT or Xquery, and apply xforms to an already translated document. As per betterFORM documentation I chose the first, and it's a mess. How does Dojo solve this? Each widget has a language attribute and can be assigned an nls object from its own namespace or anywhere else. The object is of course malleable at runtime, and if it's not available a default language is selected. As an aside, betterFORM does use the Dojo locale, but, alas, incorrectly (in 4.1).
There is no way I can have the same power in a declarative way when it comes to localization. As it turns out, the publishing company I work for published a standard work on localization. Perhaps I should read it to find my assumptions are wrong. But apart from localization I think the issue remains. Forms are widgets, rarely used in inline text. They should be approached data-centric, at least on the client. When form data needs to be validated or stored on the server it should be done in parallel with processing on the client, which in my opinion should be leading (I'm staying far away from HTML5 forms too). Forms can be easily generated from simple types and a schema. Creating a schema in a GUI is much more user-friendly than writing XML, so this is a MUST. Even if that GUI would create XForms, forcing its rules on a user is a no-go. Yes, I have ideas for such a GUI, but they are not part of this article. The topic was abandoning hope and XForms. Perhaps now it is only fair to admit that I never had much hope for XForms anyway...
It seems to me that anyone promising "programming without programmers" is just blowing smoke; he's merely created a new class of programmers. Any technology with sufficient expressive power to do what people want done is going to be isomorphic to COBOL and Ada. A computing scientist could likely show *why* this is so; meanwhile, my experience across three decades suggests strongly that it is so.
ReplyDeleteEvery few years something new comes along, promising what nobody will actually say: you can get rid of some workers who command high wages because they are good at something nobody else understands at all. It never happens; you just change out one set of experts for another. The only way to win that game is not to play, and then the competition eats you for lunch.
Concerning interaction with the user through forms, perhaps nobody has actually first stated the problem clearly and *then* designed a solution. (Heh, that would require still other expensive experts.) I think such an effort might *begin* with the realizations that you have documented above.
Thank you for a thought-provoking posting.
Just to be clear, I do share the idealism with the XForms people, and I don't care so much about the pecking order in companies. People in the XML community are exceptionally hard-working, perhaps too hard, and I feel very much connected with them. I think their goal is not to create the next best thing, but to balance between re-usability and ease of use. I do agree, however, that at the moment especially the Java community - with Apache looming behind it - is exploding. Too many tools sprawled claiming to be the next best thing in web app development. I'm not fond of beans anyway...
DeleteYet through the idealism that was the driving force behind e.g. HTML, a whole new world of development came into being, which otherwise wouldn't have, and I am thankful for that. But the Standards Committee is in trouble. I was amazed to experience at XML Prague last month that so many standards are found to be inadequate. So much so that people are inventing their respective wheels while driving the same car together. XML was found to be inadequate, XSL-FO, JavaScript (both to build clients and to query data), CSS, even RDF... It was the first time I went to such a conference, and was expecting people in the same community to cooperate more. But it seems there is a crisis there, and it's due to this crisis I think, not ego or money, that people decided to take their own course.
I think in form design the problem is clear enough, but it comes from the client (i.e. the user). Also I believe that when it comes to re-usability, there will have to be defaults that can be overwritten more easily. So actually I'm vouching for even a more strict standard to begin with. And to repeat myself, it is through the development of XForms that I see how. I'd rather see or efforts combined and not diverge and aim at a standard that is less troublesome... So here you see how stubborn the idealism really is.
I think the issue is the democratic nature of open standards: eventually the standard will be pinned down once enough smart people reach a consensus, but in practice that could take years or possibly never at all. It leaves the innovators the choice of going with an emerging standard hoping it will eventually stick, or going off on a tangent to do it your own way.
ReplyDeleteIndeed. But that doesn't even mean XForms was ever a good idea. I've just finished my form builder based on JSON Schema, but it could have been XML Schema or RelaxNG based. XForms looks like a schema, but it isn't. JSON Schema has the "format" property for supplying validation info, and in many cases this is enough to infer an form element type (or when taking some liberties with it). I found that you can infer a lot from a schema and data without extending it too much with presentation-related matters or even submission specifics. Given some defaults you don't have to repeat everything in three (or more) places, and that saves you a lot of time. Considering its time-consuming nature XForms is a half-baked product and will never become a decent standard IMO.
DeleteHi, some demos here might be of interest:
ReplyDeletehttp://www.collinta.com.au/projects.xml
Why?
DeleteWe'll put some ideas forward about the strengths and weaknesses of XForms at Balisage 2013.
ReplyDeletehttp://www.balisage.net/2013/Program.html#w445r
We hope to stimulate some healthy debate!
Dear Steve,
DeleteAs I understand it you propose a data-centric approach based on xml schema, which in turn generates xforms. I'd argue that in this angular-bracket paradigm the developer still lacks control over specific functionality, for which again you have to rely on the toolset with its inherent standards problem. If you want to stay in that box I believe a debate will proof quite useless. There is a broader problem to address, and that is that of general usability vs controllability.
Hi,
DeleteWe assume a REST API, that is 'resource' centric.
We quote from Kurt Cagle's 2008 Balisage paper at the start of our paper. “REST Oriented Architectures (ROA): Taking a resourceful approach to web data.”
But basically saying that XForms is a "data-driven programming" approach that suites data as REST 'resources', including the XForms and Schemas themselves, and relating that to MVC.
We are just reporting on ideas that have come from our experiments so far, there is much more to be done. Have a look at Kurt's paper.
Hi Steve,
DeleteA blog may not be the best place for an in-depth discussion of anything, but your comments are quite terse. May I invite you to give your point of view on using xforms in a nutshell? Thanks.
Also, I think that developing ROA is separate from the choice of client technology, and the use of xforms is far from obvious. It has never spread into the mobile world afaik, whereas OO and MVC design is widely used. I'm certainly no proponent of either, but I'm fed up with programming in angular brackets. A mixed language are far better suited imo (e.g. functional javascript and related).
DeleteYea I cannot go into details as its all in the paper and I've just spent a week working on that, so not that keen to repeat it all here. In a nutshell, XML is a very flexible means to build representations of resources, XForms is a very good way to build an online XML Editor/Reader. I see XForms as being very cheap, potentially. Hand editing XML text should be banned!
ReplyDeleteHa indeed! Thanks.
Delete