This post originally appeared on the AppSignal blog as part of a two part series.

What is Kredis?

“Keyed Redis” is a recent addition to the Rails developer’s toolkit that strives to simplify storing and accessing structured data on Redis. It’s a railtie that provides convenient wrappers to streamline working with it in three ways:

• Ruby-esque API: For example, the collection types like Kredis.list or Kredis.set emulate native Ruby types (and their respective API) as much as possible.
• Typings: Especially handy for collections, Kredis can handle type casting the elements from/to standard data types (e.g. datetime, json).
• ActiveRecord DSL: Probably the biggest asset of the library, it allows you to easily connect any Redis data structure with a specific model instance

Here’s an example from the README:

class Person < ApplicationRecord
  kredis_list :names
  kredis_unique_list :skills, limit: 2
  kredis_enum :morning, values: %w[ bright blue black ], default: "bright"
  kredis_counter :steps, expires_in: 1.hour
end

person = Person.find(5)
person.names.append "David", "Heinemeier", "Hansson" # => RPUSH people:5:names "David" "Heinemeier" "Hansson"
true == person.morning.bright?                       # => GET people:5:morning
person.morning.value = "blue"                        # => SET people:5:morning
true == person.morning.blue?                         # => GET people:5:morning

The major benefit of using Kredis is that it makes it easy to store ephemeral information associated with a certain record, but independent of the session. Typically, when you need to persist data in Rails you have a few options, of which the two most common ones are:

• ActiveRecord: In most cases this requires adding a column or otherwise patching your data model. A migration is needed, plus optional backfilling of old records.
• Session: the default key/value store of every Rails app, requires no or little setup. The downside is that data stored in it doesn’t survive a login/logout cycle.

Kredis brings a third option to the table: Little setup is required, apart from invoking the DSL in the model. But unless your Redis instance goes down, your data is stored across sessions, and even devices. A good use case for Kredis thus is uncritical information that you want to share across device borders, e.g. in a web app and a companion mobile app.

Case Study: Persist and Restore a Collapsed/Expanded State

A typical instance of such a case is the persisting of UI state, such as

• Sidebar open/closed state
• Tree view open/closed state
• Accordion collapsed/expanded state
• Custom dashboard layout
• how many lines of a data table to display

etc.

Exemplarily, we will take a look at how to manage the collapsed/expanded state of a <details> element.

Let’s start out with a fresh Rails app, add kredis to the bundle and run its installer:

$ rails new ui-state-kredis
$ cd ui-state-kredis

$ bundle add kredis
$ bin/rails kredis:install

Note: This will create a Redis configration file in config/redis/shared.yml.

For the rest of this article I will assume that you have a local running Redis instance. On Mac OS with Homebrew, this is as easy as running

$ brew install redis 

Please consult the official “Getting Started” guide for information on how to install Redis on your operating system.

User Authentication

As the entity to store UI state information on, we are going to user a User model. To avoid bikeshedding here, I will just use what Devise provides out of the box:

$ bundle add devise
$ bin/rails generate devise:install
$ bin/rails generate devise User
$ bin/rails db:migrate

We then create an example user in the Rails console:

$ bin/rails c 

irb(main):001:1* User.create(
irb(main):002:1*   email: "julian@example.com",
irb(main):003:1*   password: "mypassword",
irb(main):004:1*   password_confirmation: "mypassword"
irb(main):005:1> )

Our Example App: An Online Store

To illustrate how Kredis can help persist the state of a complex tree structure, let’s pretend we are running an online department store. To this end, we will scaffold Department and Product models. We include a self join from department to department, to create a two-level nested structure:

$ bin/rails g scaffold Department name:string department:references
$ bin/rails g scaffold Product name:string department:references
$ bin/rails db:migrate

We have to permit null parents of course, to allow for our tree structure roots:

  class CreateDepartments < ActiveRecord::Migration[7.0]
    def change
      create_table :departments do |t|
        t.string :name
-       t.references :department, null: false, foreign_key: true
+       t.references :department, foreign_key: true

        t.timestamps
      end
    end
  end

Our Department and Product models are defined as such:

class Department < ApplicationRecord
  belongs_to :parent, class_name: "Department", optional: true
  has_many :children, class_name: "Department", foreign_key: "department_id"
  has_many :products
end

class Product < ApplicationRecord
  belongs_to :department
end

Finally, we use faker to generate some seed data:

$ bundle add faker
$ bin/rails c

irb(main):001:1* 5.times do
irb(main):002:2*   Department.create(
irb(main):003:2*     name: Faker::Commerce.unique.department(max: 1),
irb(main):004:3*     children: (0..2).map do
irb(main):005:4*       Department.new(
irb(main):006:4*         name: Faker::Commerce.unique.department(max: 1),
irb(main):007:5*         products: (0..4).map do
irb(main):008:5*           Product.new(name: Faker::Commerce.unique.product_name)
irb(main):009:4*         end
irb(main):010:3*       )
irb(main):011:2*     end
irb(main):012:1*   )
irb(main):013:0> end

Scaffolding a Storefront

We’ll create a very simple HomeController that will act as our shop’s storefront.

$ bin/rails g controller Home index --no-helper
      create  app/controllers/home_controller.rb
       route  get 'home/index'
      invoke  erb
      create    app/views/home
      create    app/views/home/index.html.erb
      invoke  test_unit
      create    test/controllers/home_controller_test.rb

We perform a self join on the departments’ children to retrieve only those which actually have subdepartments (or, in other words, are our tree’s roots):

# app/controllers/home_controller.rb

class HomeController < ApplicationController
  def index
    @departments = Department.joins(:children).distinct
  end
end

In the index view, we set up a nested tree view using two levels of <details> elements for our departments:

<!-- app/views/home/index.html.erb -->

<% @departments.each do |dep| %>
  <details>
    <summary><%= dep.name %></summary>

    <% dep.children.each do |child_dep| %>
      <details style="margin-left: 1rem">
        <summary><%= child_dep.name %></summary>

        <ul>
          <% child_dep.products.each do |prod| %>
            <li><%= prod.name %></li>
          <% end %>
        </ul>
      </details>
    <% end %>
  </details>
<% end %>

Right now we have a tree view of departments with intentionally silly product names that we can explore by opening and closing:

What we’d like to accomplish is to persist the disclosure state of the individual categories, which we will tend to next.

Persisting UI State

Here is what we are going to do, step by step:

1. We will add a kredis_set called open_department_ids to the User model. The reason we are using a set here is that it doesn’t allow duplicates, so we can safely add and remove our departments.
2. We will create a UIStateController that will receive as params
   • the department_id and
   • the open state of that department.

   It will then add or remove this department to the kredis_set on the currently logged in user.

3. Create a Stimulus controller which will listen for the toggle event on the details element, and send over the respective payload.

Let’s get to it!

Adding said kredis data structure to the User model is as easy as calling kredis_set and passing an identifier:

  # app/models/user.rb

  class User < ApplicationRecord
    # Include default devise modules. Others available are:
    # :confirmable, :lockable, :timeoutable, :trackable and :omniauthable
    devise :database_authenticatable, :registerable,
           :recoverable, :rememberable, :validatable

+   kredis_set :open_department_ids
  end

Next, we generate a UIStateController to receive the UI state updates. Note that we have to configure the generated route to be a patch endpoint:

$ bin/rails g controller UIState update --no-helper --skip-template-engine
      create  app/controllers/ui_state_controller.rb
       route  get 'ui_state/update'
      invoke  test_unit
      create    test/controllers/ui_state_controller_test.rb

  Rails.application.routes.draw do
-   get 'ui_state/update'
+   patch 'ui_state/update'
    get 'home/index'
    resources :products
    resources :departments
    devise_for :users
    # Define your application routes per the DSL in https://guides.rubyonrails.org/routing.html

    # Defines the root path route ("/")
    root "home#index"
  end

In the controller there’s our first encounter with Kredis’ API. We can see that it tries to conform to Ruby developers’ expectations as close as possible, so you can add to the set using <<, and delete using remove.

# app/controllers/ui_state_controller.rb

class UiStateController < ApplicationController
  def update
    if ui_state_params[:open] == "true"
      current_user.open_department_ids << params[:department_id]
    else
      current_user.open_department_ids.remove(params[:department_id])
    end

    head :ok
  end

  private

  def ui_state_params
    params.permit(:department_id, :open)
  end
end

What’s happening here is that we toggle the presence of a specific department_id in the set based on the open param being handed over from the client. To complete the picture, we have to write some client-side code to transmit these UI state changes. We are going to use @rails/request.js to perform the actions, so we have to pin it:

$ bin/importmap pin @rails/request.js
Pinning "@rails/request.js" to https://ga.jspm.io/npm:@rails/request.js@0.0.8/src/index.js

In a new stimulus controller that’s to be attached to a specific <details> element, we append the department ID and its open state to a FormData object and submit it:

// app/javascript/controllers/ui_state_controller.js

import { Controller } from "@hotwired/stimulus";
import { patch } from "@rails/request.js";

export default class extends Controller {
  static values = {
    departmentId: Number,
  };

  async toggle() {
    const body = new FormData();
    body.append("open", this.element.open);
    body.append("department_id", this.departmentIdValue);

    await patch("/ui_state/update", {
      body,
    });
  }
}

We edit our view code as proposed, and listen for the toggle event of each <details> element to trigger the UI state updates:

  <!-- app/views/home/index.html.erb -->

  <% @departments.each do |dep| %>
-   <details>
+   <details
+     data-controller="ui-state"
+     data-action="toggle->ui-state#toggle"
+     data-ui-state-department-id-value="<%= dep.id %>"
+   >
      <summary><%= dep.name %></summary>
      <% dep.children.each do |child_dep| %>
-       <details style="margin-left: 1rem">
+       <details style="margin-left: 1rem"
+         data-controller="ui-state"
+         data-action="toggle->ui-state#toggle"
+         data-ui-state-department-id-value="<%= child_dep.id %>"
+       >
          <summary><%= child_dep.name %></summary>

          <ul>
            <% child_dep.products.each do |prod| %>
              <li><%= prod.name %></li>
            <% end %>
          </ul>
        </details>
      <% end %>
    </details>
  <% end %>

Rehydrate Manually

The only component that is missing to go full circle is rehydrating our DOM to the desired state once the user refreshes the page. We do this manually, by adding the open attribute to the <details> node if its department ID is present in the Kredis set:

  <!-- app/views/home/index.html.erb -->

  <% @departments.each do |dep| %>
    <details
      data-controller="ui-state"
      data-action="toggle->ui-state#toggle"
      data-ui-state-department-id-value="<%= dep.id %>"
+     <%= "open" if current_user.open_department_ids.include?(dep.id) %>
    >
      <summary><%= dep.name %></summary>

      <% dep.children.each do |child_dep| %>
        <details style="margin-left: 1rem"
          data-controller="ui-state"
          data-action="toggle->ui-state#toggle"
          data-ui-state-department-id-value="<%= child_dep.id %>"
+         <%= "open" if current_user.open_department_ids.include?(child_dep.id) %>
        >
          <summary><%= child_dep.name %></summary>

          <ul>
            <% child_dep.products.each do |prod| %>
              <li><%= prod.name %></li>
            <% end %>
          </ul>
        </details>
      <% end %>
    </details>
  <% end %>

Finally, here’s the result in a GIF. Note that the open/closed state of individual tree nodes is preserved over 2 levels:

A Generalized User-local Container for UI State

The great pain point with the above is that we have to invent a lot of Kredis keys. Remember that we had to concoct a Kredis set key (open_department_ids) on the user model. Now, following our example, imagine a user might be associated to the products she bought via has_many. Assume that each of these products has a details page which also has some ephemeral state attached.

All of a sudden, the complexity increases tremendously: We now have N kredis_sets to cater for, prefixed with the product ID. Or we could handle it on the Product model, storing the state for each user ID. Either way, it is going to get cumbersome, and we’ll likely have to resort to metaprogramming to smooth it out a bit.

This, you’ll agree with me, is not a delightful challenge. It would be very beneficial for the app’s architecture if we could abstract that away.

And this is precisely what we are going to attempt. We’ll use an accordion on the product details page to illustrate this case and also its resolution.

Preparation

To start, we’ll add two more text columns, description and specs, to our Product model:

$ bin/rails g migration AddDescriptionAndSpecsToProducts description:text specs:text
$ bin/rails db:migrate

Again, in the Rails console, we’ll provide some seed data:

Product.find_each do |product|
  product.update(description: Faker::Lorem.paragraph, specs: Faker::Markdown.ordered_list)
    description: Faker::Lorem.paragraph,
    specs: Faker::Markdown.ordered_list
  )
end

Finally, let’s amend the _product partial to include those two new properties in an “accordion”, again making use of the <details> element.

  <!-- app/views/products/_product.html.erb -->

  <div id="<%= dom_id product %>">
    <p>
      <strong>Name:</strong>
      <%= product.name %>
    </p>

    <p>
      <strong>Department:</strong>
      <%= product.department_id %>
    </p>

+   <p>
+     <details>
+       <summary>Description</summary>
+ 
+       <%= product.description %>
+     </details>
+   </p>
+ 
+   <p>
+     <details>
+       <summary>Specs</summary>
+ 
+       <%= simple_format product.specs %>
+     </details>
+   </p>  
  </div>

In our home index view, let’s link to the products:

  <!-- app/views/home/index.html.erb -->

  <!-- ... -->

        <ul>
          <% child_dep.products.each do |prod| %>
-           <li><%= prod.name %></li>
+           <li><%= link_to prod.name, prod %></li>
          <% end %>
        </ul>

  <!-- ... -->

Here the detail view this produces:

Observing UI Changes with JavaScript

Now we have to rewrite our UIState Stimulus controllers to listen for arbitrary DOM changes. We do this by placing a MutationObserver on its element, which looks for attribute changes after it connects. In its callback (mutateState), the following happens:

1. All attributes on the element, except for the data-controller attribute itself, are collected,
2. Their respective values are stored on an attributes object with their name as the key.
3. This object is appended to a new FormData object, along with a unique key that identifies the exact instance of the UI state (we’ll get to that in a moment).
4. This is then sent over to the server as before, in the form of a PATCH request.

  // app/javascript/controllers/ui_state_controller.js

  import { Controller } from "@hotwired/stimulus";
- import { patch } from "@rails/request.js";
+ import { get, patch } from "@rails/request.js";

  export default class extends Controller {
    static values = {
-     departmentId: String,
+     key: String,
    };

-   async toggle() {
-     const body = new FormData();
-     body.append("open", this.element.open);
-     body.append("department_id", this.departmentIdValue);
-  
-     await patch("/ui_state/update", {
-       body,
-     });
-   }

+   async connect() {
+     this.mutationObserver = new MutationObserver(this.mutateState.bind(this));
+
+     this.mutationObserver.observe(this.element, { attributes: true });
+   }
+
+   disconnect() {
+     this.mutationObserver?.disconnect();
+   }
+
+   async mutateState() {
+     const body = new FormData();
+
+     const attributes = {};
+     this.element
+       .getAttributeNames()
+       .filter((name) => name !== "data-controller")
+       .map((name) => {
+         attributes[name] = this.element.getAttribute(name);
+       });
+
+     body.append("key", this.keyValue);
+     body.append("attributes", JSON.stringify(attributes));
+
+     await patch("/ui_state/update", {
+       body,
+     });
+   }
  }

Persisting UI State

To generalize the persisting of UI state, we have to accomplish two things:

1. Generate Kredis keys that are unique regarding user, resource (in our case the product), and the location in the view respectively,
2. Use these keys to persist the attributes object received from the client.

Using Rails Helpers

So first, we have to solve the problem of coming up with unique keys. If you look at the requirements carefully, they read a lot like the built-in Rails fragment cache helper.

In short, this helper will build a cache key based on the template it’s called from and any cacheable Ruby objects passed to it. To quote the documentation:

views/template/action:7a1156131a6928cb0026877f8b749ac9/projects/123
      ^template path   ^template tree digest             ^class   ^id

We don’t need the cache helper though, because we are not actually storing a fragment, we just need a key. Thankfully, the method it calls internally to generate one, cache_fragment_name, is also exposed.

Let’s check out how we can make use of this. If, in our products/_product.html.erb partial we add the following line:

<%= cache_fragment_name([current_user, product]) %>

Something like this is rendered:

["products/_product:0e55db8bd12c9b268798d6550447b303",
  [#<User id: 1, email: "julian@example.com", ...>,
  #<Product id: 68, name: "Durable Wool Wallet", ...>]
]

Clearly this nested array needs some massaging, but we are already very close. What’s still missing is a reference to the specific line in the template, but we can obtain this from the call stack’s first entry representing a template. Let’s extract this to a helper and convert it to a string:

# app/helpers/application_helper.rb

module ApplicationHelper
  def ui_state_key(name)
    key = cache_fragment_name(name, skip_digest: true)
      .flatten
      .compact
      .map(&:cache_key)
      .join(":")

    key += ":#{caller.find { _1 =~ /html/ }}"

    key
  end
end

Note: We are skipping the view digesting as including the caller location makes it redundant.

Called as ui_state_key([current_user, product]), this yields a string of the following format:

"users/1:products/68:/usr/app/ui-state-kredis/app/views/products/_product.html.erb:23:in `_app_views_products__product_html_erb___2602651701187259549_284440'"

Now there’s one final piece to solve: Because we are going to include this key in our markup (to be picked up by the Stimulus controller above), we’ll want to obfuscate it. Otherwise it would be easy for a bad actor to exchange the user ID and/or the product global ID using the browser’s developer tools. A simple digest will do in our case, because we do not intend to decrypt it back.

  # app/helpers/application_helper.rb

  module ApplicationHelper
    def ui_state_key(name)
      key = cache_fragment_name(name, skip_digest: true)
        .flatten
        .compact
        .map(&:cache_key)
        .join(":")

      key += ":#{caller.first}"

-     key
+     ActiveSupport::Digest.hexdigest(key)
    end
  end

That’s it, now we can obtain a digest of our UI fragment that will be unique to the location in the template and it’s MTIME, as well as user and resource:

"d45028686c0171e1e6e8a8ab78aae835"

Equipped with this, we can now attach the Stimulus controllers along with the respective keys to our <details> elements:

  <!-- app/views/products/_product.html.erb -->

  <div id="<%= dom_id product %>">
    <p>
      <strong>Name:</strong>
      <%= product.name %>
    </p>

    <p>
      <strong>Department:</strong>
      <%= product.department_id %>
    </p>

    <p>
      <details
+       data-controller="ui-state"
+       data-ui-state-key-value="<%= ui_state_key([current_user, product]) %>"
      >
        <summary>Description</summary>

        <%= product.description %>
      </details>
    </p>

    <p>
      <details
+       data-controller="ui-state"
+       data-ui-state-key-value="<%= ui_state_key([current_user, product]) %>"
      >
        <summary>Specs</summary>

        <%= simple_format product.specs %>
      </details>
    </p>  
  </div>

We can now set out to generalize the UIStateController as well. For this, we obtain a new Kredis.json key using the transmitted key param. Whenever an update of UI state occurs, all the attributes sent over as form data will be parsed and stored therein.

  # app/controllers/ui_state_controller.rb

  class UiStateController < ApplicationController
+   include ActionView::Helpers::SanitizeHelper

+   before_action :set_ui_state

    def update
-     if ui_state_params[:open] == "true"
-       current_user.open_department_ids << params[:department_id]
-     else
-       current_user.open_department_ids.remove(params[:department_id])
-     end
+     @ui_state.value = JSON.parse(params[:attributes]).deep_transform_values { sanitize(_1) }

      head :ok
    end

    private

    def ui_state_params
-     params.permit(:department_id, :open)
+     params.permit(:attributes, :key)
    end

+   def set_ui_state
+     @ui_state = Kredis.json params[:key]
+   end
  end

Rehydrating the UI

Since our UI state is now safely stored in an ephemeral Kredis key, the only piece of the puzzle that’s missing is how to put the UI back together. This process is called rehydration. In this example, we’ll use server-side rehydration.

For this, we will revisit the ui_state_key helper from above, and extend it with the rehydration logic.

   # app/helpers/application_helper.rb
   module ApplicationHelper
+    def remember_ui_state_for(name, &block)
+      included_html = capture(&block).to_s.strip
+      fragment = Nokogiri::HTML.fragment(included_html)
+    
+      first_fragment_child = fragment.first_element_child
+    
+      # rehydrate
+      ui_state = Kredis.json ui_state_key(name)
+    
+      ui_state.value&.each do |attribute_name, value|
+        first_fragment_child[attribute_name] = sanitize(value, tags: [])
+      end
+    
+      # add stimulus controller and create unique key
+      first_fragment_child["data-controller"] = "#{first_fragment_child["data-controller"]} ui-state".strip
+      first_fragment_child["data-ui-state-key-value"] = ui_state_key(name)
+    
+      first_fragment_child.to_html.html_safe
+    end 

     def ui_state_key(name)
       key = cache_fragment_name(name, skip_digest: true)
         .flatten
         .compact
         .map(&:cache_key)
         .join(":")

       key += ":#{caller.find { _1 =~ /html/ }}"

       key
     end
   end

The remember_ui_state_for helper takes a name argument that it will pass on to the ui_state_key helper from above. Before it does that, though, it will capture the HTML from the block passed to it and extract its first fragment child with Nokogiri.

Afterwards, the actual rehydration logic starts:

1. the ui_state is read back from Kredis
2. in case such a key already exists, each attribute is put back on the HTML element obtained in the previous step. To prevent any XSS vulnerabilities, the attributes’ values are also sanitized
3. finally, the Stimulus controller is attached to the element, along with the ui_state_key.

Any change to the attributes would now again be transmitted by the MutationObserver, and any page reload would invoke this helper again, rehydrating them upon the element.

Note: An additional security mechanism would be safelisting the allowed attributes, which I have excluded from this example for simplicity.

In our product partial, instead of wiring up the Stimulus controller manually, we must now use the new view helper:

  <!-- app/views/products/_product.html.erb -->

  <div id="<%= dom_id product %>">
    <p>
      <strong>Name:</strong>
      <%= product.name %>
    </p>

    <p>
      <strong>Department:</strong>
      <%= product.department_id %>
    </p>

    <p>
-     <details
-       data-controller="ui-state"
-       data-ui-state-key-value="<%= ui_state_key([current_user, product]) %>"
-     >
-       <summary>Description</summary>
-
-       <%= product.description %>
-     </details>
+     <%= remember_ui_state_for([current_user, product]) do %>
+       <details>
+         <summary>Description</summary>
+ 
+         <%= product.description %>
+       </details>
+     <% end %>
    </p>

    <p>
-     <details
-       data-controller="ui-state"
-       data-ui-state-key-value="<%= ui_state_key([current_user, product]) %>"
-     >
-       <summary>Specs</summary>
-
-       <%= simple_format product.specs %>
-     </details>
+     <%= remember_ui_state_for([current_user, product]) do %>
+       <details >
+         <summary>Specs</summary>
+  
+         <%= simple_format product.specs %>
+       </details>
+     <% end %>
    </p>  
  </div>

This is what the end result looks like:

Conclusion

Starting out from a brief description of what Kredis can and cannot do, we have developed an example use case for storing ephemeral UI state using a bespoke Redis key. Afterwards, we have identified the drawbacks and came up with a generalized solution that can be applied to any DOM node whose state of attributes we want to track on the server side.

It turns out that this approach is perfect for HTML elements or custom web components that reflect their state in one or more attributes (e.g. <details>, Shoelace, Lit, etc.).

Finally, let me point out that these considerations inspired me to develop a library called solder, which I would be happy to receive feedback on!