Ocsigen

Home

Doc

Eliom

Js_of_ocaml

Server

Lwt

Tyxml

Start

Warning: Reason support is experimental. We are looking for beta-tester and contributors.

Client server reactive application with Ocsigen

This is a short tutorial showing how to implement a simple reactive client-server application using Js_of_ocaml, Eliom and Ocsigen Start.

We are going to implement an application that can display a list of messages and that allows connected users to add new messages.

You will learn:

  • How to use Ocsigen Start to quickly build an application with user management.
  • How to create a client-server reactive interface: the HTML is generated indifferently server-side or client-side, and contains reactive parts that are updated automatically when the data change.
  • How to implement a notification system for your application. Users are notified when a new item (a message in our case) arrives.

First step: a basic application with user management

Ocsigen Start contains a set of higher level libraries for Eliom (user management, tips, notifications). It also contains a template for eliom-distillery that creates an application with user management. You can use this template as a starting point for your project. 

eliom-distillery -name tutoreact -template os.pgocaml

This template is using PostgreSQL to store the data. You need a recent version of Postgresql installed on your system. With that available, you can create the local database and start the database server: 

make db-init
make db-create
make db-schema

Compile and run the program: 

make test.byte

Go to http://localhost:8080, you should see the welcome page. You can now register a user and log in. Because the send mail function is not configured, the activation links will be printed on the console you started the server with.

At any point, if you want to get back to this tutorial later, you may need to start the database again: 

make db-start

While doing this tutorial, if you plan to work on another Ocsigen project requiring a database, do not forget to stop the tutorial's database beforehand: 

make db-stop

Display messages from db

To make this example more realistic, let's suppose that we do not want to display all the messages in the database, but only a few of them.

In this tutorial, we will not focus on the implementation details of the database part. Create a new file named tutoreact_messages.eliom. From now on, if not explicitely specified, the code we are going to write will go there. We are going to create a module Db containing these functions: 

val get_messages : unit -> int list Lwt.t
val get_message : int -> string Lwt.t
val add_message : string -> int Lwt.t

You can try to make your own implementation using for instance pgocaml. Here's our implementation using Ocsipersist: 

[%%server
module Db = struct
  let db = Ocsipersist.Polymorphic.open_table "messages"

  let last_key =
    Eliom_reference.eref
      ~persistent:"index"
      ~scope:Eliom_common.global_scope (-1)

  let get_message id =
    let%lwt db = db in
    Ocsipersist.Polymorphic.find db (string_of_int id)

  let get_messages () =
    let%lwt index = Eliom_reference.get last_key in
    let rec aux n l = if n > index then l else aux (n+1) (n::l) in
    Lwt.return (aux 0 [])

  let lock = Lwt_mutex.create ()

  let add_message v =
    let%lwt () = Lwt_mutex.lock lock in
    let%lwt index = Eliom_reference.get last_key in
    let index = index + 1 in
    let%lwt () = Eliom_reference.set last_key index in
    Lwt_mutex.unlock lock;
    let%lwt db = db in
    let%lwt () = Ocsipersist.Polymorphic.add db (string_of_int index) v in
    Lwt.return index
end]

Add the following code: 

[%%shared
    open Eliom_content.Html
    open Eliom_content.Html.D]
let%server display userid_o =
  let%lwt messages = Db.get_messages () in
  let%lwt l =
    Lwt_list.map_s
      (fun id ->
        let%lwt msg = Db.get_message id in
        Lwt.return (li [txt msg]))
      messages
  in
  Lwt.return [ul l]

Depending on your database, it is probably more efficient to fetch all messages and their identifiers using only one request. Here we use Lwt_list.map_s to do the requests sequentially.

The content of the main page is defined in the file tutoreact_handlers.eliom. Replace the code of main_service_handler by: 

let%server main_service_handler userid_o () () =
  let%lwt content = Tutoreact_messages.display userid_o in
  Tutoreact_container.page userid_o content

The main_service_handler you just replaced was in a shared section. Therefore, we also need to change two other files to take into consideration this modification.

In the file tutoreact_handlers.eliomi, move the definition of main_service_handler from the shared section to the server section.

In the file tutoreact.eliom, move the registration of main_service from the shared section to the server section.

Try to compile in order to see if everything is fine.

Adding new messages

Add an input in the page, for connected users

To add an input in the page for connected users, replace the function display by the following version: 

let%server display_messages () =
  let%lwt messages = Db.get_messages () in
  let%lwt l =
    Lwt_list.map_s
      (fun id ->
        let%lwt msg = Db.get_message id in
        Lwt.return (li [txt msg]))
      messages
  in
  Lwt.return (ul l)

let%server display userid_o =
  let%lwt messages = display_messages () in
  let l =
    match userid_o with
    | None -> []
    | _ -> [Raw.input ~a:[a_input_type `Text;
                          a_style "border-style:solid"] ()]
  in
  Lwt.return (messages :: l)

Make function Db.add_message accessible from the client

To be able to call a function from the client-side program, use let%rpc: 

let%rpc add_message (value : string) : unit Lwt.t =
  let%lwt _ = Os_current_user.get_current_userid () (* fails if not connected *) in
  Db.add_message value

The parameter [%json: string] describes the type of the function parameter. This exhibits the syntax provided by ppx_deriving extended with our JSON plugin. We use this for safe server-side unmarshalling of data sent by the client.

Bind the input to call the function

To call the function from the client program, we will define a client value, a client-side expression that is accessible server-side. The client value will be executed client-side after the page is loaded. The syntax for client values of type t is [%client (... : t)].

Replace the second branch of the match in the function display by: 

let inp = Raw.input ~a:[a_input_type `Text; a_style "border-style:solid"] () in
let _ = [%client
  (let open Js_of_ocaml_lwt.Lwt_js_events in
   let inp = To_dom.of_input ~%inp in
   async (fun () -> changes inp (fun _ _ ->
     let value = Js_of_ocaml.Js.to_string inp##.value in
     inp##.value := Js_of_ocaml.Js.string "";
     let%lwt _ = add_message value in
     Lwt.return ()))
   : unit)
] in
[inp]
  • We use module Lwt_js_events to manage events.
  • The syntax ~%v allows using a server-side value v client-side.
  • To_dom.of_input returns the JS element corresponding to the OCaml value ~%inp.
  • Lwt_js_events.async is similar to Lwt.async.
  • obj##.a allows the access to the field a of the JavaScript object obj (see Js_of_ocaml PPX extension).
  • changes takes a JS element and a function that will be executed every time a "change" event is received on this element.

This function gets the value of the input, resets the content of the input, and calls our server-side function. Do not forget the conversions between OCaml strings and JS strings since they are different!

Compile and run the program again. Now the messages should be added in the database whenever you use the input. However you need to refresh the page to display them.

Structure of a client-server application

We have seen how to send data to the server without stopping the client-side program. Now we want to automatically update the page when new messages are sent. Generally, the main difference between a web application and a website is that in the case of a web application, a client-side program runs and persists accross HTTP calls (remote procedure calls or page changes). The client process must be able to receive notifications from the server and update the page accordingly, without regenerating it entirely. It is common practice to generate the full interface client-side. But this is not suitable for all cases. It is usually better to keep the old-style web interaction and generate pages server-side, for example to enable search engine indexing. In this tutorial, we will see how to generate pages indifferently (and with the same code) from both sides.

In this section, we will see how to implement this kind of applications very concisely thanks to three notions:

  • The client-server cache of data
  • Reactive pages
  • Notification system

You will be able to test once you finish the three following sections!

Client-server cache

The module Eliom_cscache implements a cache of data, an association table where you will put the data of your application client-side. For the sake of uniformity (as we want to use it in shared sections), the cache is also implemented server-side, with scope "request". This avoids retrieving the same data from the database twice for the same request.

We create a client-server cache by calling the function Eliom_cscache.create server-side. The server-side cache cache created by this function will be accessible client-side through an injection ~%cache.

We implement a function get_data to fetch the data from the database. This function must have an implementation both server-side and client-side: 

let%rpc get_data (id : int) : string Lwt.t = Db.get_message id
let%server cache : (int, string) Eliom_cscache.t =
  Eliom_cscache.create ()

Reactive interface

Updating the interface when some data change is usually not straightforward. This is usually done by putting identifiers on elements to find them, and manually modifying page elements using low-level JS functions.

A very elegant solution to simplify this consists in using Functional Reactive Programming (FRP). In reactive programming, you define relations between different pieces of data once, and each update automatically produces the recomputation of all the dependent data. In Ocsigen we use the module React combined with ReactiveData, which extends React to deal with incremental updates in lists. Have a look at the documentation of the above modules if you are not familiar with FRP.

The client-side module Eliom_content.Html.R enables defining reactive page elements.

The module Eliom_shared enables defining shared reactive signals server-side. In order to do that, we use shared values, values defined both server-side and client-side. The server-side module Eliom_content.Html.R enables constructing HTML5 elements that get updated automatically based on the signals of Eliom_shared. The modules Eliom_shared.React and Eliom_shared.ReactiveData implement interfaces very similar to React and ReactiveData, but operate on shared signals.

Implementation of the reactive interface

display_message now needs to be implemented in a shared fashion and take its data from the cache. In order to do that, we call Eliom_cscache.find cache get_data key from either side to get the value associated to key. If the value is not present in the cache, it will be fetched using the function get_data and added to the cache. 

let%shared display_message id =
  let%lwt msg = Eliom_cscache.find ~%cache get_data id in
  Lwt.return (li [txt msg])

The function display_messages now creates a reactive list of message identifiers, and maps page content from this reactive value using module Eliom_shared.ReactiveData. Note that rmessage is a tuple, the first element is the list, the second element is the update function. 

let%server display_messages () =
  let%lwt messages = Db.get_messages () in
  let rmessages = Eliom_shared.ReactiveData.RList.create messages in
  let%lwt content =
    Eliom_shared.ReactiveData.RList.Lwt.map_p
      [%shared display_message ]
      (fst rmessages)
  in
  Lwt.return (R.ul content)

Notifications

We now want to be notified when a message has been added. To do that easily, we use the module Os_notif from Ocsigen Start.

We first define a notification module for the type of data we want clients to be able to listen on (here the lists of message identifiers): 

[%%server
module Forum_notif = Os_notif.Make_Simple (struct
  type key = unit
  type notification = int
end)
]

key is the type of the identifier of the data we want to listen on. In our case, there is a single message list (thus unit suffices as the identifier since we don't need to be specific).

notification is the type of the notifications to send. Here: the identifier of the new message to be added in the list.

We define a function to handle notifications. It adds the new identifier in the reactive list of messages: 

let%client handle_notif_message_list rmessages (_, msgid) =
  Eliom_shared.ReactiveData.RList.cons msgid (snd rmessages)

We notify the server that we are listening on this piece of data by calling Forum_notif.listen (server-side). Notifications are received client-side through a React event Forum_notif.client_ev (). We map this event to the function handle_notif_message_list, meaning that we will execute this function when this event happens. 

let%server display_messages () =
  Forum_notif.listen ();
  let%lwt messages = Db.get_messages () in
  let rmessages = Eliom_shared.ReactiveData.RList.create messages in
  ignore [%client
    (ignore
       (React.E.map (handle_notif_message_list ~%rmessages)
          ~%(Forum_notif.client_ev () : (unit * int) Eliom_react.Down.t))
     : unit)
  ];
  let%lwt content =
    Eliom_shared.ReactiveData.RList.Lwt.map_p
      [%shared display_message ]
      (fst rmessages)
  in
  Lwt.return (R.ul content)

When we add a message, we notify all the clients listening on this piece of data: 

let%rpc add_message (value : string) : unit Lwt.t =
   let%lwt id = Db.add_message value in
   Forum_notif.notify () id;
   Lwt.return ()

The program is now fully functional, you can now test it! You should see the messages being added without the need to reload the page, even if messages are added by another user! Try with several browser windows.

More information on cache and client-server reactive data

In this section we will demonstrate additional Eliom functionality for client-server programming by implementing some new features in our forum:

  • Multi-page forum
  • Client-side spinner while loading data

Multi-page forum

We now want a forum with several pages, located at URLs http://localhost:8080/i, where i represents the forumid as an integer.

Services

In the file tutoreact_services.eliom, we define the new following service: 

let%server forum_service =
  Eliom_service.create
    ~path:(Eliom_service.Path [""])
    ~meth:(Eliom_service.Get
            (Eliom_parameter.(suffix (int "i"))))
    ()

In the file tutoreact_services.eliomi, we define its signature, do not forget to put it in a server section: 

[%%server.start]

val forum_service
  : ( int
    , unit
    , Eliom_service.get
    , Eliom_service.att
    , Eliom_service.non_co
    , Eliom_service.non_ext
    , Eliom_service.reg
    , [`WithSuffix]
    , [`One of int] Eliom_parameter.param_name
    , unit
    , Eliom_service.non_ocaml )
    Eliom_service.t

In the file tutoreact_handlers.eliom, we define the handler we will associate to our new service: 

let%server forum_service_handler userid_o forumid () =
  let%lwt content = Tutoreact_messages.display userid_o forumid in
  Tutoreact_container.page userid_o content

In the file tutoreact_handlers.eliomi, we define its signature, in the server section: 

val forum_service_handler
  :  Os_types.User.id option
  -> int
  -> unit
  -> Os_page.content Lwt.t

In the file tutoreact.eliom, we register our handler to our new service in the server section: 

Tutoreact_base.App.register ~service:Tutoreact_services.forum_service
    (Tutoreact_page.Opt.connected_page Tutoreact_handlers.forum_service_handler)

Since we have a new parameter forumid, we need to take it into consideration in many places.

In the file tutoreact_messages.eliom, the functions display_messages and display take it as a new parameter. Do not forget to also replace the latter in the call of display_messages in function display: 

let%server display_messages forumid =
  ...
let%server display userid_o forumid =
  let%lwt messages = display_messages forumid in
  ...

In the file tutoreact_handlers.eliom, update the code of main_service_handler: 

let%server main_service_handler forumid userid_o () () =
  let%lwt content = Tutoreact_messages.display userid_o forumid in
  Tutoreact_container.page userid_o content

In the file tutoreact_handlers.eliomi, update its signature: 

val main_service_handler
  :  int
  -> Os_types.User.id option
  -> unit
  -> unit
  -> Os_page.content Lwt.t

In the file tutoreact.eliom, in the main_service, we have to specify the forumid of the forum we want to reach when we arrive in our application. We will take 0 for instance and give it as the parameter of main_service_handler. We update the registration of main_service: 

Tutoreact_base.App.register
    ~service:Os_services.main_service
    (Tutoreact_page.Opt.connected_page
    (Tutoreact_handlers.main_service_handler 0));

Db

The functions Db.get_messages and Db.add_message now take the forum identifier: 

[%%server
module Db = struct

  let db = Ocsipersist.Polymorphic.open_table "messages"

  let dbf = Ocsipersist.Polymorphic.open_table "forums"

  let last_key =
    Eliom_reference.eref
      ~persistent:"index" ~scope:Eliom_common.global_scope (-1)

  let get_message id =
    let%lwt db = db in
    Ocsipersist.Polymorphic.find db (string_of_int id)

  let get_messages forumid =
    let%lwt dbf = dbf in
    try%lwt
      Ocsipersist.Polymorphic.find dbf (string_of_int forumid)
    with Not_found ->
      Lwt.return []

  let add_message forumid v =
    let%lwt index = Eliom_reference.get last_key in
    let index = index + 1 in
    let%lwt () = Eliom_reference.set last_key index in
    let%lwt db = db in
    let%lwt () = Ocsipersist.Polymorphic.add db (string_of_int index) v in
    let%lwt l = get_messages forumid in
    let%lwt dbf = dbf in
    let%lwt () =
      Ocsipersist.Polymorphic.add dbf
        (string_of_int forumid)
        (index :: l)
    in
    Lwt.return index

end
]

Message type

Since we are now adding besides the message, the forumid as well in our database, we need to specify a new type: 

[%%shared
    type add_message_type = int * string [@@deriving json]
]

We don't forget to take that into consideration in the function add_message. 

let%rpc add_message ((forumid, value) : add_message_type) : unit Lwt.t =
   let%lwt id = Db.add_message forumid value in
   Forum_notif.notify () id;
   Lwt.return ()

In the function display, in the client section: 

...
  add_message (~%forumid, value)
...

Cache of forum message identifiers

We must send the notifications only to the clients listening on the same forum.

We will create a new client-server cache to keep the reactive list of message identifiers for each forums: 

let%server forumcache :
  (int,
   int Eliom_shared.ReactiveData.RList.t *
   int Eliom_shared.ReactiveData.RList.handle) Eliom_cscache.t =
  Eliom_cscache.create ()

We will now implement the equivalent of get_data for this new cache.

Be very careful:

In get_data_forum, we must find the reactive list of messages in the new cache —if it exists — instead of creating a new one! Otherwise you will have several reactive data for the same forum and the page updates will fail!

To do that, we provide an optional argument ?default to the function Eliom_shared.ReactiveData.RList.create, a client value (optionally) containing the current reactive list. If it does not exist in the cache, a new one will be created like previously: 

let%rpc get_data_forum (forumid : int) : _ Lwt.t =
  let%lwt messages = Db.get_messages forumid in
  let default = [%client
    ((try Some (Eliom_cscache.find_if_ready ~%forumcache ~%forumid)
      with _ -> None)
     : 'a option)
  ] in
  Lwt.return (Eliom_shared.ReactiveData.RList.create ~default messages)

display_messages now takes the reactive list from the cache: 

let%server display_messages forumid =
  Forum_notif.listen ();
  let%lwt rmessages =
    Eliom_cscache.find forumcache get_data_forum forumid
  in
  ...

Notifications dependent on forum ID

Since we now want to be specific about the data we want to listen to, the unit parameter we defined can't be used anymore. Indeed, notifications now depend on the identifier. We want to receive notifications only for the forums present in the client-side cache of forums. Therefore, we just change the type key of module Forum_notif to use an integer (instead of unit): 

[%%server
module Forum_notif = Os_notif.Make_Simple (struct
  type key = int
  type notification = int
end)
]

The function Forum_notif.notify used in the function add_message now takes the forumid parameter. 

let%rpc add_message ... =
  ...
  Forum_notif.notify forumid id;
  ...

In the function display_messages, we need to take care of the forumid parameter and the type annotation of client_ev: 

let%server display_messages forumid =
  Forum_notif.listen (forumid : int);
  ...
  ~%(Forum_notif.client_ev () : (int * int) Eliom_react.Down.t))
  ...

We annotate the type of forumid in the call of the function listen to help the typing system.

The function handle_notif_message now takes the reactive list rmessage from the cache, therefore we no longer need it as a parameter: 

let%client handle_notif_message_list (forumid, msgid) =
  try
    let rmessages = Eliom_cscache.find_if_ready ~%forumcache forumid in
    Eliom_shared.ReactiveData.RList.cons msgid (snd rmessages)
  with Not_found | Eliom_cscache.Not_ready -> ()

In the function display_messages, do not forget to remove the injection of rmessage in the call of handle_notif_message_list in the client section: 

...
(React.E.map handle_notif_message_list
...

Display a spinner while loading the messages

Retrieving messages from server can take time. To display a spinner while loading the messages when you send them, replace the function display_message by: 

let%shared display_message id =
  let th =
    let%lwt msg = Eliom_cscache.find ~%cache get_data id in
    Lwt.return [div [txt msg]]
  in
  let%lwt v = Ot_spinner.with_spinner th in
  Lwt.return (li [v])

To simulate network latency, you can add a Lwt_unix.sleep in the server-side get_data function. 

let%server get_data id =
  let%lwt () = Lwt_unix.sleep 2.0 in
  Db.get_message id

The full code (tutoreact_messages.eliom):

[%%shared
    open Eliom_content.Html
    open Eliom_content.Html.D
]
[%%server
module Db = struct
  let db = Ocsipersist.Polymorphic.open_table "messages"
  let dbf = Ocsipersist.Polymorphic.open_table "forums"

  let last_key =
    Eliom_reference.eref ~persistent:"index" ~scope:Eliom_common.global_scope
      (-1)

  let get_message id =
    let%lwt db = db in
    Ocsipersist.Polymorphic.find db (string_of_int id)

  let get_messages forumid =
    let%lwt dbf = dbf in
    try%lwt Ocsipersist.Polymorphic.find dbf (string_of_int forumid)
    with Not_found -> Lwt.return []

  let add_message forumid v =
    let%lwt index = Eliom_reference.get last_key in
    let index = index + 1 in
    let%lwt () = Eliom_reference.set last_key index in
    let%lwt db = db in
    let%lwt () = Ocsipersist.Polymorphic.add db (string_of_int index) v in
    let%lwt l = get_messages forumid in
    let%lwt dbf = dbf in
    let%lwt () =
      Ocsipersist.Polymorphic.add dbf (string_of_int forumid) (index :: l)
    in
    Lwt.return index

end

module Forum_notif = Os_notif.Make_Simple (struct
  type key = int
  type notification = int
end)
]
[%%shared
    type add_message_type = int * string [@@deriving json]
]
let%rpc add_message ((forumid, value) : add_message_type) : unit Lwt.t =
   let%lwt id = Db.add_message forumid value in
   Forum_notif.notify forumid (id : int);
   Lwt.return ()
let%server cache : (int, string) Eliom_cscache.t = Eliom_cscache.create ()

let%server forumcache :
  (int,
   int Eliom_shared.ReactiveData.RList.t *
   int Eliom_shared.ReactiveData.RList.handle) Eliom_cscache.t =
  Eliom_cscache.create ()
ler%rpc get_data (id : int) : string Lwt.t =
  let%lwt () = Lwt_unix.sleep 2.0 in
  Db.get_message id
let%rpc get_data_forum (forumid : int) : _ Lwt.t =
  let%lwt messages = Db.get_messages forumid in
  let default = [%client
    (try
       Some (Eliom_cscache.find_if_ready ~%forumcache ~%forumid)
     with _ ->
       None
       : 'a option)
  ] in
  Lwt.return (Eliom_shared.ReactiveData.RList.create ~default messages)
let%shared display_message id =
  let th =
    let%lwt msg = Eliom_cscache.find ~%cache get_data id in
    Lwt.return [div [txt msg]]
  in
  let%lwt v = Ot_spinner.with_spinner th in
  Lwt.return (li [v])
let%client handle_notif_message_list (forumid, msgid) =
  try
    let rmessages = Eliom_cscache.find_if_ready ~%forumcache forumid in
    Eliom_shared.ReactiveData.RList.cons msgid (snd rmessages)
  with Not_found | Eliom_cscache.Not_ready -> ()
let%server display_messages forumid =
  Forum_notif.listen (forumid : int);
  let%lwt rmessages =
    Eliom_cscache.find forumcache get_data_forum forumid
  in
  ignore [%client
    (ignore
       (React.E.map handle_notif_message_list
          ~%(Forum_notif.client_ev () : (int * int) Eliom_react.Down.t))
     : unit)
  ];
  let%lwt content =
    Eliom_shared.ReactiveData.RList.Lwt.map_p
      [%shared display_message]
      (fst rmessages)
  in
  Lwt.return (R.ul content)

let%server display userid_o forumid =
  let%lwt messages = display_messages forumid in
  let l =
    match userid_o with
    | None -> []
    | _ ->
        let inp =
          Raw.input ~a:[a_input_type `Text; a_style "border-style:solid"] ()
        in
        let _ =
          [%client
            (let open Js_of_ocaml_lwt.Lwt_js_events in
             let inp = To_dom.of_input ~%inp in
             async (fun () ->
                 changes inp (fun _ _ ->
                     let value = Js_of_ocaml.Js.to_string inp##.value in
                     inp##.value := Js_of_ocaml.Js.string "";
                     let%lwt _ = add_message (~%forumid, value) in
                     Lwt.return ()))
              : unit)]
        in
        [inp]
  in
  Lwt.return (messages :: l)