Serialization using Play JSON

§Play-JSON Serialization

To enable JSON Serialization there are three steps you need to follow.

The first step is to define your Format for each class that is to be serialized, this can be done using automated mapping or manual mapping.

implicit val format: Format[ItemAdded] = Json.format

Best practice is to define the Format as an implicit in the classes companion object, so that it can be found by implicit resolution.

The second step is to implement JsonSerializerRegistry and have all the service formats returned from its serializers method.

import com.lightbend.lagom.scaladsl.playjson.JsonSerializer
import com.lightbend.lagom.scaladsl.playjson.JsonSerializerRegistry

object MyRegistry extends JsonSerializerRegistry {
  override val serializers = Vector(
    JsonSerializer[ItemAdded],
    JsonSerializer[OrderAdded]
  )
}

Having done that, you can provide the serializer registry by overriding the jsonSerializerRegistry component method in your application cake, for example:

import com.lightbend.lagom.scaladsl.server._
import com.lightbend.lagom.scaladsl.cluster.ClusterComponents

abstract class MyApplication(context: LagomApplicationContext)
    extends LagomApplication(context)
    with ClusterComponents {
  override lazy val jsonSerializerRegistry = MyRegistry
}

If you need to use the registry outside of a Lagom application, for example, in tests, this can be done by customising the creation of the actor system, for example:

import akka.actor.ActorSystem
import akka.actor.setup.ActorSystemSetup
import com.lightbend.lagom.scaladsl.playjson.JsonSerializerRegistry

val system = ActorSystem(
  "my-actor-system",
  ActorSystemSetup(
    JsonSerializerRegistry.serializationSetupFor(MyRegistry)
  )
)

§Compression

Compression, as described here, is only used for persistent events, persistent snapshots and remote messages with the service cluster.

JSON can be rather verbose and for large messages it can be beneficial to enable compression. That is done by using the JsonSerializer.compressed[T] builder method instead of the JsonSerializer.apply[T] (as shown in the example snippet above):

import com.lightbend.lagom.scaladsl.playjson.JsonSerializer
import com.lightbend.lagom.scaladsl.playjson.JsonSerializerRegistry

object RegistryWithCompression extends JsonSerializerRegistry {
  override val serializers = Vector(
    // 'ItemAdded' uses `apply[T]()` .
    JsonSerializer[ItemAdded],
    // The OrderAdded message is usually rather big, so we want it compressed
    // when it's too large.
    JsonSerializer.compressed[OrderAdded]
  )
}

The serializer will by default only compress messages that are larger than 32Kb. This threshold can be changed with configuration property:

lagom.serialization.json {

  # The serializer will compress the payload if the message class
  # was registered using JsonSerializer.compressed and the payload
  # is larger than this value. Only used for remote messages within
  # the cluster of the service.
  compress-larger-than = 32 KiB

}

§Automated mapping

The Json.format[MyClass] macro will inspect a case class for what fields it contains and produce a Format that uses the field names and types of the class in the resulting JSON.

The macro allows for defining formats based on the exact structure of the class which is handy and avoids spending development time on explicitly defining the format, on the other hand it tightly couples the structure of the JSON with the structure of the class so that a refactoring of the class unexpectedly leads to the format being unable to read JSON that was serialized before the change. There are tools in place to deal with this (see schema evolution) but care must be taken.

If the class contains fields of complex types, it pulls those in from implicit marked Formats in the scope. This means that you must provide such implicit formats for all the complex types used inside a class before calling the macro.

case class UserMetadata(twitterHandle: String)
object UserMetadata {
  implicit val format: Format[UserMetadata] = Json.format
}
case class AddComment(userId: String, comment: String, userMetadata: UserMetadata)
object AddComment {
  implicit val format: Format[AddComment] = Json.format
}

§Manual mapping

Defining a Format can be done in several ways using the Play JSON APIs, either using JSON Combinators or by manually implementing functions that turn a JsValue into a JsSuccess(T) or a JsFailure().

case class OrderAdded(productId: String, quantity: Int)

import play.api.libs.functional.syntax._
import play.api.libs.json._

object OrderAdded {
  implicit val format: Format[OrderAdded] =
    (JsPath \ "product_id")
      .format[String]
      .and((JsPath \ "quantity").format[Int])
      .apply(OrderAdded.apply, unlift(OrderAdded.unapply))
}

§Special mapping considerations

§Mapping options

The automatic mapping will handle Option fields, for manual mapping of optional fields you can use (JsPath \ "optionalField").formatNullable[A]. This will treat missing fields as None allowing for adding of new fields without providing an explicit schema migration step.

§Mapping singletons

For toplevel singletons (Scala objects) you can use com.lightbend.lagom.scaladsl.playjson.Serializers.emptySingletonFormat to get a Format that outputs empty JSON (as the type is also encoded along side the data).

case object GetOrders {
  implicit val format: Format[GetOrders.type] =
    JsonSerializer.emptySingletonFormat(GetOrders)
}

§Mapping hierarchies

When mapping a hierarchy of types, for example an ADT, or a trait or abstract class you will need to provide a Format for the supertype, that based on some information in the JSON decides which subtype to deserialize.

import play.api.libs.json._

sealed trait Fruit
case object Pear                 extends Fruit
case object Apple                extends Fruit
case class Banana(ripe: Boolean) extends Fruit

object Banana {
  implicit val format: Format[Banana] = Json.format
}

object Fruit {
  implicit val format = Format[Fruit](
    Reads { js =>
      // use the fruitType field to determine how to deserialize
      val fruitType = (JsPath \ "fruitType").read[String].reads(js)
      fruitType.fold(
        errors => JsError("fruitType undefined or incorrect"), {
          case "pear"   => JsSuccess(Pear)
          case "apple"  => JsSuccess(Apple)
          case "banana" => (JsPath \ "data").read[Banana].reads(js)
        }
      )
    },
    Writes {
      case Pear  => JsObject(Seq("fruitType" -> JsString("pear")))
      case Apple => JsObject(Seq("fruitType" -> JsString("apple")))
      case b: Banana =>
        JsObject(
          Seq(
            "fruitType" -> JsString("banana"),
            "data"      -> Banana.format.writes(b)
          )
        )
    }
  )
}

§Schema Evolution

When working on long running projects using Akka Persistence Typed, Lagom Persistence (classic), or any kind of Event Sourcing, schema evolution becomes an important aspect of developing your application. The requirements as well as our own understanding of the business domain may (and will) change over time.

Lagom provides a way to perform transformations of the JSON tree model during deserialization. To do those transformations you can either modify the json imperatively or use the Play JSON transformers

We will look at a few scenarios of how the classes may be evolved.

§Remove Field

Removing a field can be done without any migration code. Both manual and automatic mappings will ignore properties that does not exist in the class.

§Add Field

Adding an optional field can be done without any migration code if automated mapping is used or manual mapping is used and you have made sure a missing field is read as a None by your format (see mapping options).

Old class:

case class ItemAdded(shoppingCartId: String, productId: String, quantity: Int)

New class with a new optional discount property:

case class ItemAdded(shoppingCartId: String, productId: String, quantity: Int, discount: Option[BigDecimal])

Let’s say we want to have a mandatory discount property without default value instead:

case class ItemAdded(shoppingCartId: String, productId: String, quantity: Int, discount: Double)

To add a new mandatory field we have to use a JSON migration adding a default value to the JSON

This is how a migration logic would look like for adding a discount field using imperative code:

class ShopSerializerRegistry extends JsonSerializerRegistry {
  import play.api.libs.json._

  override val serializers = ShopCommands.serializers ++ ShopEvents.serializers

  private val itemAddedMigration = new JsonMigration(2) {
    override def transform(fromVersion: Int, json: JsObject): JsObject = {
      if (fromVersion < 2) {
        json + ("discount" -> JsNumber(0.0d))
      } else {
        json
      }
    }
  }

  override def migrations = Map[String, JsonMigration](
    classOf[ItemAdded].getName -> itemAddedMigration
  )
}

Create a concrete subclass of JsonMigration handing it the current version of the schema as a parameter, then implement the transformation logic on the JsObject in the transform method when an older fromVersion is passed in.

Then provide your JsonMigration together with the classname of the class that it migrates in the migrations map from your JsonSerializerRegistry.

Alternatively you can use the Play JSON transformers API which is more concise but arguably has a much higher threshold to learn.

class ShopSerializerRegistry extends JsonSerializerRegistry {
  import play.api.libs.json._

  override val serializers = ShopCommands.serializers ++ ShopEvents.serializers

  val addDefaultDiscount = JsPath.json.update((JsPath \ "discount").json.put(JsNumber(0.0d)))

  override def migrations = Map[String, JsonMigration](
    JsonMigrations.transform[ItemAdded](
      immutable.SortedMap(
        1 -> addDefaultDiscount
      )
    )
  )
}

In this case we give the JsonMigrations.transform method the type it is for, and a sorted map of transformations that has happened leading up to the current version of the schema.

§Rename Field

Let’s say that we want to rename the productId field to itemId in the previous example.

case class ItemAdded(shoppingCartId: String, itemId: String, quantity: Int)

The imperative migration code would look like:

private val itemAddedMigration = new JsonMigration(2) {
  override def transform(fromVersion: Int, json: JsObject): JsObject = {
    if (fromVersion < 2) {
      val productId = (JsPath \ "productId").read[JsString].reads(json).get
      json + ("itemId" -> productId) - "productId"
    } else {
      json
    }
  }
}

override def migrations = Map[String, JsonMigration](
  classOf[ItemAdded].getName -> itemAddedMigration
)

And alternatively the transformer based migration:

val productIdToItemId =
  JsPath.json
    .update(
      (JsPath \ "itemId").json.copyFrom((JsPath \ "productId").json.pick)
    )
    .andThen((JsPath \ "productId").json.prune)

override def migrations = Map[String, JsonMigration](
  JsonMigrations.transform[ItemAdded](
    immutable.SortedMap(
      1 -> productIdToItemId
    )
  )
)

§Structural Changes

In a similar way we can do arbitrary structural changes.

Old class:

case class Customer(name: String, street: String, city: String, zipCode: String, country: String)

New classes:

case class Address(street: String, city: String, zipCode: String, country: String)

case class Customer(name: String, address: Address, shippingAddress: Option[Address])

The migration code could look like:

import play.api.libs.json._
import play.api.libs.functional.syntax._

val customerMigration = new JsonMigration(2) {

  // use arbitrary logic to parse an Address
  // out of the old schema
  val readOldAddress: Reads[Address] = {
    (JsPath \ "street")
      .read[String]
      .and(
        (JsPath \ "city").read[String])
      .and(
        (JsPath \ "zipCode").read[String])
      .and(
        (JsPath \ "country").read[String])(Address)
  }

  override def transform(fromVersion: Int, json: JsObject): JsObject = {
    if (fromVersion < 2) {
      val address           = readOldAddress.reads(json).get
      val withoutOldAddress = json - "street" - "city" - "zipCode" - "country"

      // use existing formatter to write the address in the new schema
      withoutOldAddress + ("address" -> Customer.addressFormat.writes(address))
    } else {
      json
    }
  }
}

override def migrations: Map[String, JsonMigration] = Map(
  classOf[Customer].getName -> customerMigration
)

§Rename Class

It is also possible to rename the class. For example, let’s rename OrderAdded to OrderPlaced.

Old class:

case class OrderAdded(shoppingCartId: String)

New class:

case class OrderPlaced(shoppingCartId: String)

The migration code would look like:

override def migrations: Map[String, JsonMigration] = Map(
  JsonMigrations
    .renamed(fromClassName = "com.lightbend.lagom.shop.OrderAdded", inVersion = 2, toClass = classOf[OrderPlaced])
)

When a class has both been renamed and had other changes over time the name change is added separately as in the example and the transformations are defined for the new class name in the migrations map. The Lagom serialization logic will first look for name changes, and then use the changed name to resolve any schema migrations that will be done using the changed name.