Querydsl 2.0 has been released!

Querydsl 2.0 has been released! After two alpha and two beta releases we are proud to announce the final 2.0 release. The biggest single change in Querydsl 2.0 compared to previous 1.* releases is the improved Expression API. We got rid of the coupling between the query construction contract and the DSL features of the Expression API.

The Expression API is now divided into two parts, an interface based part which describes the archetypes of Expressions which are used in Query construction, and a class based part, which provides the standard Expression DSL.

Here is an UML overview of the new Expression model : Expression model changes

In addition to the improvements of the Expression API there is improved Eclipse IDE support, initial Scala and Mongodb support.

Querying with Scala

Scala support for Querydsl has been on the roadmap for some time, and with the Querydsl 2.0 release we got initial support for using Querydsl in Scala applications released.

The goal was to use the Query interfaces as such, and to provide an alternative expression DSL better suited for Scala usage. With most Querydsl modules APT is used for code generation. As there is no equivalent for Scala we didn't try to provide code generation features for Scala domain models.

Instead we combined the Alias-feature of the Querydsl Core module with the Scala expression DSL. The result is in our opinion quite compact and readable.

For the moment Querydsl with Scala can be used with JPA, JDO and Mongodb. To get started, you will need the additional querydsl-scala dependency in your project in addition to the dependencies for your Querydsl backend support. Querydsl for Scala has been tested against Scala 2.8.0.

Using Maven you would need to declare the following additional dependency :

<dependency>
      <groupId>com.mysema.querydsl</groupId>
      <artifactId>querydsl-scala</artifactId>
      <version>${querydsl.version}</version>
</dependency>

Here is a minimal example with JPA/Hibernate :

Domain model :

@Entity
class User {
  @BeanProperty
  @Id
  var id: Integer = _;
  @BeanProperty
  var userName: String = _;
  @BeanProperty
  @ManyToOne
  var department: Department = _;
}

@Entity
class Department {
  @BeanProperty
  @Id
  var id: Integer = _;
  @BeanProperty
  var name: String = _;
}

And here are some query examples

import com.mysema.query.scala.Conversions._
import com.mysema.query.jpa.impl.JPAQuery

import com.mysema.query.types.path._
import org.junit.Test

class JPAQueryTest {

  val person = alias(classOf[Person])

  @Test
  def Various() {
    // list
    query from person where (person.firstName $like "Rob%") 
      .list person

    // unique result
    query from person where (person.firstName $like "Rob%") 
      .unique person

    // long where
    query from person 
      .where (person.firstName $like "Rob%", person.lastName $like "An%") 
      .list person

    // order
    query from person orderBy (person.firstName asc) list person

    // not null 
    query from person 
      .where (person.firstName $isEmpty, person.lastName $isNotNull) 
      .list person
  }

  def query() = new JPAQuery(entityManager)

}

The main import for Querydsl Scala integration is the following

import com.mysema.query.scala.Conversions._

The factory method for query creation is

def query() = new JPAQuery(entityManager)

In addition to queries you need variables which can be created like this

var person = alias(classOf[Person])

The person variable is a proxy instance of the the Person class which can be used in queries. Now you can construct your queries, populate them via from-where-...-orderBy calls and get the projection out via list/uniqueResult/listResults calls.

Querydsl expressions are constructed via method calls starting with the "$" sign.

With the Querydsl Java API a simple like expression would be constructed like this :

person.firstName.like("Rob%")

Using the Scala API it is

person.firstName $like "Rob%"

For more examples see src/test/scala in SVN.

Secure Registration Method

Many services require users to register using real email addresses. Usually this is done so that the system sends a confirmation message to the email address after the user has registered. This requires a two phase flow in which the account details are first persisted into an inactive state and later activated. An alternative is to make the account temporarily active to let the user in right away, but close it after a while if there is no email confirmation.

There are several drawbacks to this two phase registration method:

1. Increased complexity in handling account activation


2. Unconfirmed users are persisted and need to be cleaned up from the database from time to time


3. If the user is let in temporarily right away, there have to be additional measures to guard against bots and to authorize actions that are reserved for actually confirmed accounts.


4. It is very inconvenient for the user if she first gives all her details and then somehow fails to confirm the account, e.g. due to a typo in the email address.

A much simpler, yet secure way of registration starts by confirming the email address. First the user gives her email address and the system sends her an email containing a unique registration link. With a properly constructed link, nothing needs to be persisted on the server side at this point. Only when the user follows the link, is she requested for other account details. The account is then created directly into an active state and the user is let in with full access rights.

The trick is in the link itself. It should be constructed to be a) hard to guess and tamper with and b) expiring by itself after a fixed amount of time. This can be achieved easily by adding a timestamp and the email address itself as parameters to the link and using a Message Authentication Code (MAC) to ensure that these parameters are not tampered with:

https://yourdomain.com/register
?timestamp=123456789&email=john.doe@example.com&mac=MAC

in which the MAC is constructed using a hash function like SHA:

sha(123456789;john.doe@example.com;SECRET)

Here the SECRET is a long random phrase that is only known by the server. Even if one knows the building blocks of the link, one cannot fake it unless one knows the secret phrase.

Before even showing the user a registration form, the MAC is to be checked by comparing the supplied MAC with one calculated from the other message parameters.

Once we know that we can trust the link parameters we know that the given email address exists and is accessible by the user. Expiring the link is simply a matter of checking whether the timestamp is not too far in the past (e.g. three hours).

Software business would be so easy without customers and users

This has sometimes come to my mind when being in one of those moments of pressure to deliver the promised features on time. Or when hearing that the delivery is not what was ordered or doesn’t have all the features imagined or is perfect apart from being late.

Well, no-one says the software business is easy, and it definitely isn’t so. But why is creating working software in the real world so difficult? I think the main reason is because the software business is about creating and selling something which is much more intangible than what we as human beings are used to deal with. And we are pretty much just learning how this new field of industry really works.

The intangible nature of the software can be seen from how

• things can change fast
• big changes are possible
• it’s not easy to describe what is actually needed
• it’s not easy to say if something is good enough
• there is always multiple workable ways to solve problems
  

The rules and the laws on how to build a good house and good software are different, we have a long history on successfully constructing the buildings, we know how to do it well, but we are just learning on what kind of new skills and ways of thinking are needed to create successful software projects.

Embracing the change

Using agile methods to create software comes from the learned lesson that it’s not useful to fight against the intangible nature of software and the process of developing it. We have to have tools that are able embrace the change rather than fight against it. And the whole iterative development process with practices like continuous integration, delivering often, test driven development and so on are aimed to make that possible.

So now we have a toolbox full of good ways to react fast when things go into unexpected directions as they always do and everything is fine, right?

Well, no. What hasn’t changed that much at all is the relationship between the customers and developers. While the methods of creating the software have changed to be more agile, the contracts have not changed that much. And in the end the contract is the bottom line of what is possible in the relationship between these two parties.

The tighter the contract, the less it allows room to react, and the more there is risk that the whole project is doomed in the end due to trying to pretend that what everyone is doing is something solid, pre-designable and predictable far in advance.

The trust allows more space to react

Constructing software for someone is about fusing their ideas and wishes into working software. And if it would only be that, it would be quite manageable, but unfortunately there is no escape from the hard facts of time and money, and usually in the form of not having enough of both.

Agility means the ability to change the direction fast, the steer of the boat of software creation. And changing direction means there has to be some openness that the current direction might not be the best direction. So being agile means inherently being able to withstand a little bit of unsureness, the lack of knowing. The more we can feel confident in moving to uncharted territory with the knowledge that if things go bad, we can react correctly, the faster we can travel safely. But that’s confidence on the tools and the people you are traveling with, it’s not the (contractually) enforced confidence in the plan that was created in the past and which might not be working very well anymore in the changed situation.

So, in order for the software process to be truly agile, there has to be trust between the parties. Trust that both are actually traveling together, not a totally known path, but mutually agreed direction. When this trust is not present, what remains are the rigid steel frames of contracts. And usually this state also contracts the space of creativity too.

The big questions

Now that we have some ideas on what is a good way to create software and what kind of relationship between a paying customer and supplier would be good base for the successful project, there are two big questions:

• How do you sell an agile software project?
• How do you create a good contract for an agile project?
  

I’m positive that there is plenty of room for improvements in that arena and it’s going to be interesting to see how the agile movement is going to be seen in the arena of selling and making contracts. I will try to cover some ideas on them in my next blog post.