[{"_1":2,"_147":-5,"_148":-5},"loaderData",{"_3":4,"_30":31,"_145":-5,"_146":-5},"routes/_main",{"_5":6},"socialMedia",[7,12,15,18,21,24,27],{"_8":9,"_10":11},"link","mailto:cedar.onyxes0q@icloud.com","icon","mail",{"_8":13,"_10":14},"https://github.com/sjdonado","github",{"_8":16,"_10":17},"rss.xml","rss",{"_8":19,"_10":20},"https://linkedin.com/in/sjdonado","linkedin",{"_8":22,"_10":23},"https://slides.com/sjdonado","presentation",{"_8":25,"_10":26},"https://stackoverflow.com/users/8108856/juan-rodriguez","layers",{"_8":28,"_10":29},"https://vsco.co/sjdonado/gallery","camera","routes/_main.posts",{"_32":33},"posts",[34,51,63,73,85,94,105,115,125,134],{"_35":36,"_37":38,"_39":40,"_41":42,"_8":43,"_44":45,"_47":48},"id",108,"title","You are the easiest person to fool","date","3/9/2025","description","We humans are remarkably good at deceiving ourselves.","/posts/you-are-the-easiest-person-to-fool","tags",[46],"life","content",{"_49":50},"rendered","\n

\n
The first principle is that you must not fool yourself—and you are the easiest person to fool. — Richard Feynman, Cargo Cult Science, Caltech commencement address, 1974.¹
\n

\n\n\n\n

At the heart of Feynman’s warning lies a simple yet profound truth: we humans are remarkably good at deceiving ourselves. Our brains didn’t evolve primarily to discover objective truth, but to help us survive and sometimes that means prioritizing comfortable beliefs over accurate ones.

\n\n\n\n

This self-deception works through everyday mental shortcuts: we naturally seek out information that confirms what we already believe; we expertly weave narratives that make our self-interested decisions seem logical; and perhaps most dangerously, we convince ourselves we’re being objective when we’re anything but. What makes these processes so powerful is that they operate largely outside our awareness.

\n\n\n\n

Feynman recognized that this presents a special challenge to knowledge because we apply different standards to different sources. We’re quick to question claims from others while giving our own thinking process a free pass. This happens because we have insider access to our own motivations; we can craft justifications perfectly tailored to soothe our psychological defenses.

\n\n\n\n

Today, these cognitive blind spots are systematically amplified. Digital platforms optimize for engagement rather than accuracy, creating spaces where our biases are reinforced rather than challenged. Political divisions intensify as we interpret opposing viewpoints through increasingly hostile filters. Scientific questions become matters of identity rather than evidence.

\n\n\n\n

Science itself developed its methods: peer review, replication requirements, statistical controls; precisely as defenses against these human tendencies. Yet these safeguards only work when we acknowledge how easily we can fool ourselves.

\n\n\n\n

Feynman’s insight is not just about good science, it offers a fundamental approach to understanding reality: true knowledge begins with humility about our own mental limitations.

\n\n\n

The first principle is that you must not fool yourself—and you are the easiest person to fool. So you have to be very careful about that. After you’ve not fooled yourself, it’s easy not to fool other scientists. You just have to be honest in a conventional way after that. https://calteches.library.caltech.edu/51/2/CargoCult.htm ↩︎

",{"_35":52,"_37":53,"_39":54,"_41":55,"_8":56,"_44":57,"_47":61},103,"Migrate remix eslint-config to ESLint v9","3/2/2025","Custom ESLint v9 flat config for Remix from scratch.","/posts/migrate-remix-eslint-config-to-eslint-v9",[58,59,60],"remix","tech","tooling",{"_49":62},"\n

This weekend I finally found time to upgrade the old deprecated @remix-run/eslint-config in my Remix projects (not yet migrated to React Router v7).

\n\n\n\n

I first attempted to remove the package entirely and adopt the recommended configuration from the official Remix template (https://github.com/remix-run/remix/blob/main/templates/remix/.eslintrc.cjs). However, after running the ESLint migration tool (https://eslint.org/docs/latest/use/getting-started) npm init @eslint/migrate-config, the output was problematic. This was likely because some dependencies weren’t yet compatible with ESLint’s new flat configuration system.

\n\n\n\n

Instead, I opted for a manual approach, installing and configuring each plugin individually. Here’s the outcome:

\n\n\n\n

Install new dependencies

\n\n\n\n

npm install --save-dev @eslint/js @remix-run/dev @typescript-eslint/eslint-plugin @typescript-eslint/parser eslint eslint-plugin-react eslint-plugin-simple-import-sort eslint-plugin-tailwindcss globals tailwindcss typescript-eslint

\n\n\n\n

Changes in the dev dependencies should look similar to these:

\n\n\n\n

   \"devDependencies\": {\n+    \"@eslint/js\": \"^9.20.0\",\n-    \"@remix-run/dev\": \"^2.13.1\",\n-    \"@remix-run/eslint-config\": \"^2.13.1\",\n+    \"@remix-run/dev\": \"^2.15.3\",\n@@ -89,16 +89,22 @@\n+    \"@typescript-eslint/eslint-plugin\": \"^8.23.0\",\n+    \"@typescript-eslint/parser\": \"^8.23.0\",\n-    \"eslint\": \"^9.3.0\",\n+    \"eslint\": \"^9.20.0\",\n+    \"eslint-plugin-react\": \"^7.37.4\",\n+    \"globals\": \"^15.14.0\",\n+    \"typescript-eslint\": \"^8.23.0\",\n   },

\n\n\n\n

Migrate old config file

\n\n\n\n

Now we can remove .eslintrc.cjs and replace it with the new flat config eslint.config.js:

\n\n\n\n

import jseslint from \"@eslint/js\";\nimport reactPlugin from \"eslint-plugin-react\";\nimport globals from \"globals\";\nimport tseslint from \"typescript-eslint\";\n\n/** @type {import('eslint').Linter.Config[]} */\nexport default [\n  { files: [\"**/*.{js,mjs,cjs,ts,jsx,tsx}\"] },\n  { languageOptions: { globals: { ...globals.browser, ...globals.node } } },\n  jseslint.configs.recommended,\n  ...tseslint.configs.recommended,\n  reactPlugin.configs.flat.recommended,\n  reactPlugin.configs.flat[\"jsx-runtime\"]\n];

\n\n\n\n

Optionally, this is how it looks like if you would like to include the eslint-plugin-tailwindcss and eslint-plugin-simple-import-sort plugins:

\n\n\n\n

import jseslint from \"@eslint/js\";\nimport reactPlugin from \"eslint-plugin-react\";\nimport simpleImportSort from \"eslint-plugin-simple-import-sort\";\nimport globals from \"globals\";\nimport tseslint from \"typescript-eslint\";\nimport tailwind from \"eslint-plugin-tailwindcss\";\n\n/** @type {import('eslint').Linter.Config[]} */\nexport default [\n  { files: [\"**/*.{js,mjs,cjs,ts,jsx,tsx}\"] },\n  { languageOptions: { globals: { ...globals.browser, ...globals.node } } },\n  jseslint.configs.recommended,\n  ...tseslint.configs.recommended,\n  reactPlugin.configs.flat.recommended,\n  reactPlugin.configs.flat[\"jsx-runtime\"],\n  {\n    plugins: {\n      \"simple-import-sort\": simpleImportSort,\n    },\n    rules: {\n      \"simple-import-sort/imports\": \"error\",\n      \"simple-import-sort/exports\": \"error\",\n    },\n  },\n  ...tailwind.configs[\"flat/recommended\"],\n];

\n",{"_35":64,"_37":65,"_39":66,"_41":67,"_8":68,"_44":69,"_47":71},91,"Managing 10k+ Visitors Overnight with Bare Metal","11/13/2024","","/posts/how-bare-metal-simplicity-handled-10k-visitors-overnight-without-breaking-the-bank",[70,59],"devops",{"_49":72},"\n

\n\n\n\n

Yesterday, I Don’t Have Spotify reached #2 on Hacker News, getting 700+ points https://news.ycombinator.com/item?id=42110877. This brought in 13k views and over 10k unique visitors, pretty intense for a single day!

\n\n\n\n

What really stood out to me was how seamlessly the stack: HTMX, Bun, SQLite, and Dokku handled this traffic spike. No downtime, no slowdowns, no surprises. Just reliable performance.

\n\n\n\n

This highlights how far efficient, lightweight tools can take you. Instead of over-engineering with costly solutions, this setup handled a massive traffic spike overnight while keeping hosting costs under the price of a coffee per month.

\n\n\n\n

Sometimes, simplicity outshines the hype around “scale-ready” architectures.

\n\n\n\n

For anyone learning about infrastructure or preparing to launch a product, I thought this was worth sharing: scaling isn’t always about Kubernetes or microservices. You don’t need to invest heavily in infrastructure to achieve great results. The bare-metal approach can effectively handle real-world challenges, enabling quick and efficient scaling while keeping costs low. It’s a practical and accessible path to foster innovation.

\n\n\n\n

I Don’t Have Spotify is open source and available here: https://github.com/sjdonado/idonthavespotify. Feel free to explore, contribute, or use it as inspiration for your own projects.

\n",{"_35":74,"_37":75,"_39":76,"_41":77,"_8":78,"_44":79,"_47":83},47,"Sitemap generation by streaming from WordPress headless","11/3/2024","Optimize sitemap generation by leveraging streaming and chunked transfer encoding.","/posts/sitemap-generation-by-streaming-from-wordpress-headless",[80,81,59,82],"javascript","performance","wordpress",{"_49":84},"\n

The sitemap.xml file is very useful for search engines to understand which URLs are available for crawling. Therefore, when building a blog, generating this file can help boost SEO.

\n\n\n\n

Taking this website as an example, the sitemap includes static URLs like /home, /projects, as well as blog posts at /posts/:slug. This is the output we need to generate:

\n\n\n\n

<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<urlset xmlns=\"http://www.sitemaps.org/schemas/sitemap/0.9\">\n  <url>\n    <loc>http://localhost:5173/</loc>\n    <lastmod>2024-11-04T06:32:35.319Z</lastmod>\n    <changefreq>monthly</changefreq>\n    <priority>1</priority>\n  </url>\n  <url>\n    <loc>http://localhost:5173/home</loc>\n    <lastmod>2024-11-04T06:32:35.319Z</lastmod>\n    <changefreq>monthly</changefreq>\n    <priority>0.8</priority>\n  </url>\n  <url>\n    <loc>http://localhost:5173/posts</loc>\n    <lastmod>2024-11-04T06:32:35.319Z</lastmod>\n    <changefreq>weekly</changefreq>\n    <priority>0.8</priority>\n  </url>\n  <url>\n    <loc>http://localhost:5173/projects</loc>\n    <lastmod>2024-11-04T06:32:35.319Z</lastmod>\n    <changefreq>monthly</changefreq>\n    <priority>0.8</priority>\n  </url>\n  <url>\n    <loc>http://localhost:5173/social</loc>\n    <lastmod>2024-11-04T06:32:35.319Z</lastmod>\n    <changefreq>monthly</changefreq>\n    <priority>0.6</priority>\n  </url>\n  <url>\n    <loc>http://localhost:5173/slides</loc>\n    <lastmod>2024-11-04T06:32:35.319Z</lastmod>\n    <changefreq>monthly</changefreq>\n    <priority>0.6</priority>\n  </url>\n  <url>\n    <loc>https://sjdonado.com/posts/should-we-leave-behind-the-joy-of-curating-our-own-content</loc>\n    <lastmod>2024-11-02T12:38:11.000Z</lastmod>\n    <changefreq>weekly</changefreq>\n    <priority>0.8</priority>\n  </url>\n  <url>\n    <loc>https://sjdonado.com/posts/building-a-fast-and-compact-sqlite-cache-store</loc>\n    <lastmod>2024-10-31T17:06:21.000Z</lastmod>\n    <changefreq>weekly</changefreq>\n    <priority>0.8</priority>\n  </url>\n  <url>\n    <loc>https://sjdonado.com/posts/validated-forms-with-usefetcher-in-remix</loc>\n    <lastmod>2024-10-31T15:15:14.000Z</lastmod>\n    <changefreq>weekly</changefreq>\n    <priority>0.8</priority>\n  </url>\n  <url>\n    <loc>https://sjdonado.com/posts/fast-intuitive-smart-restaurant-search-engine-with-cloudflare-ai</loc>\n    <lastmod>2024-10-31T15:31:28.000Z</lastmod>\n    <changefreq>weekly</changefreq>\n    <priority>0.8</priority>\n  </url>\n  <url>\n    <loc>https://sjdonado.com/posts/htmx-with-bun-a-real-world-app</loc>\n    <lastmod>2024-11-03T14:02:46.000Z</lastmod>\n    <changefreq>weekly</changefreq>\n    <priority>0.8</priority>\n  </url>\n  <url>\n    <loc>https://sjdonado.com/posts/tower-of-hanoi-in-p5-js-wasm</loc>\n    <lastmod>2024-11-03T12:58:11.000Z</lastmod>\n    <changefreq>weekly</changefreq>\n    <priority>0.8</priority>\n  </url>\n  <url>\n    <loc>https://sjdonado.com/posts/self-hosted-password-manager-with-dokku</loc>\n    <lastmod>2024-10-31T15:55:16.000Z</lastmod>\n    <changefreq>weekly</changefreq>\n    <priority>0.8</priority>\n  </url>\n</urlset>

\n\n\n\n

As you might guess, the challenge of generating this file lies in how to fetch the posts using the WordPress REST API. Initially, we could opt to fetch all posts with GET /wp-json/wp/v2/posts and iterate over the response in a separate backend:

\n\n\n\n

  const response = await fetch(`${siteUrl}/wp-json/wp/v2/posts?per_page=100&_embed`);\n  const posts = await response.json();\n\n  let sitemap = '<?xml version=\"1.0\" encoding=\"UTF-8\"?>\\n';\n  sitemap += '<urlset xmlns=\"http://www.sitemaps.org/schemas/sitemap/0.9\">\\n';\n\n  for (const url of staticUrls) {\n    sitemap += `\n      <url>\n        <loc>${url.loc}</loc>\n        <lastmod>${url.lastmod}</lastmod>\n        <changefreq>${url.changefreq}</changefreq>\n        <priority>${url.priority}</priority>\n      </url>\n    `;\n  }\n\n  for (const post of posts) {\n    sitemap += `\n      <url>\n        <loc>${siteUrl}/posts/${post.slug}</loc>\n        <lastmod>${new Date(post.modified || post.date).toISOString()}</lastmod>\n        <changefreq>weekly</changefreq>\n        <priority>0.8</priority>\n      </url>\n    `;\n  }\n\n  sitemap += '</urlset>';\n\n  return new Response(sitemap, {\n    headers: {\n      'Content-Type': 'application/xml',\n    },\n  });

\n\n\n\n

That approach works fine, and in the end, the most time-consuming operation is waiting for the API response, which tends to increase linearly as more posts are fetched. The question now is, can this be faster?

\n\n\n\n

The initial answer has to be yes, since we are adding overhead by not rendering this file directly from the WordPress environment, which is closest to the database. Moving from a headless approach to creating a WP action would be the ideal, fastest scenario. However, we don’t want to do that, we want to maintain a separation of concerns, where WP is our CMS, and the blog routes are rendered elsewhere.

\n\n\n\n

This approach leads us to consider rendering the sitemap file immediately as data is received from the database. Instead of waiting for the entire dataset, we can process posts in chunks as they are sent from the database. By setting up a new REST endpoint that returns posts as chunked data, a ReadableStream can handle each chunk, appending it to the XML sitemap while streaming the file directly to the browser. This way, the browser starts receiving the sitemap file almost instantly, and the XML is built dynamically as new data arrives.

\n\n\n\n

It may sound a bit complicated, but think of it this way: We are streaming from the database to the browser by converting each post into a urlset entry in the final XML file, each post goes through two streams:

\n\n\n\n

WP (stream 1, chunk post) -> Backend (stream 2, chunk urlset) -> Browser

\n\n\n

With this in mind, let’s dive into the code.

\n\n\n\n

Stream 1: WordPress endpoint

\n\n\n\n

Chunked transfer encoding allows chunks to be sent and received independently of one another. Therefore, we query the posts in batches, collect their information, and send them one by one in chunks. We need to ensure that each JSON-encoded chunk includes its size appended at the end.

\n\n\n\n

function stream_posts() {\n    // Set the HTTP headers to indicate a stream\n    header('Content-Type: application/json');\n    header('Transfer-Encoding: chunked');\n    header('Connection: keep-alive');\n    header('X-Accel-Buffering: no'); // Disable buffering in Nginx if applicable\n\n    // Start output buffering and flush\n    ob_start();\n\n    // Fetch posts in batches\n    $query = new WP_Query([\n        'post_type' => 'post',\n        'posts_per_page' => 10,\n    ]);\n\n    while ($query->have_posts()) {\n        $query->the_post();\n\n        $post_data = [\n            'title'       => get_the_title(),\n            'link'        => get_permalink(),\n            'excerpt'     => get_the_excerpt(),\n            'date'        => get_the_date('c'),\n            'modified'    => get_the_modified_date('c'),\n            'content'     => apply_filters('the_content', get_the_content()),\n        ];\n\n        $json_chunk = json_encode($post_data) . \"\\n\";\n\n        // Output the chunk size in hexadecimal, followed by \\r\\n\n        echo dechex(strlen($json_chunk)) . \"\\r\\n\";\n\n        // Output the chunk data\n        echo $json_chunk . \"\\r\\n\";\n\n        // Flush the output buffer to send data immediately\n        ob_flush();\n        flush();\n    }\n\n    // Send the final chunk to indicate the end of the response\n    echo \"0\\r\\n\\r\\n\";\n\n    // Clean up\n    wp_reset_postdata();\n    ob_end_flush();\n\n    exit; // Terminate the script to prevent WordPress from adding any further output\n}

\n\n\n\n

Stream 2: XML file

\n\n\n\n

We do the same as the WP endpoint, but on the blog rendering side. We open the second stream to the client while keeping the first stream connection open:

\n\n\n\n

const sitemapStream = new ReadableStream({\n    async start(controller) {\n      controller.enqueue(\n        `<?xml version=\"1.0\" encoding=\"UTF-8\"?>\\n` +\n          `<urlset xmlns=\"http://www.sitemaps.org/schemas/sitemap/0.9\">\\n`\n      );\n\n      for (const url of staticUrls) {\n        const staticUrlItem = `\n          <url>\n            <loc>${url.loc}</loc>\n            <lastmod>${url.lastmod}</lastmod>\n            <changefreq>${url.changefreq}</changefreq>\n            <priority>${url.priority}</priority>\n          </url>\n        `;\n        controller.enqueue(staticUrlItem);\n      }\n\n      for await (const post of streamFromWordpress('/stream-posts')) {\n        const sitemapItem = `\n          <url>\n            <loc>${post.link}</loc>\n            <lastmod>${new Date(post.modified || post.date).toISOString()}</lastmod>\n            <changefreq>weekly</changefreq>\n            <priority>0.8</priority>\n          </url>\n        `;\n        controller.enqueue(sitemapItem);\n      }\n\n      controller.enqueue('</urlset>');\n      controller.close();\n    },\n  });\n\n  return new Response(sitemapStream, {\n    headers: {\n      'Content-Type': 'application/xml',\n    },\n  });

\n\n\n\n

And that’s it. The result is an average reduction of more than 100ms.

\n\n\n\n

\"\" — Fetching All Posts vs. Streaming Posts.

\"\" — Fetching All Posts vs. Streaming Posts.

\n\n\n\n

\n",{"_35":86,"_37":87,"_39":88,"_41":89,"_8":90,"_44":91,"_47":92},43,"Should we leave behind the joy of curating our own content?","8/2/2024","There is joy in the journey more than the destination.","/posts/should-we-leave-behind-the-joy-of-curating-our-own-content",[46,59],{"_49":93},"\n

For a while, I’ve been reading blog posts, articles, and comment threads about various tech and finance topics. I initially found it convenient to have a Medium subscription, as I thought it would allow me to have a feed automatically shaped by my interests. However, I ended up reading articles prioritized by clickbait, echo chambers and bias.

So, I canceled my subscription and switched to Hacker News. I have discovered amazing independent writers discussing fascinating topics. Eventually, I started collecting these RSS feeds in one place, creating my own “Medium”.

I believe that’s the true purpose of the internet: to be free to decide where to go and what to explore. There is joy in the journey more than the destination.

If you’re curious about the blogs I follow, you can download and import the OPML file into your RSS reader of choice.

\n",{"_35":95,"_37":96,"_39":97,"_41":98,"_8":99,"_44":100,"_47":103},1,"Building a Fast and Compact SQLite Cache Store","7/24/2024","Discover why bun:sqlite, combined with Msgpackr serialization, offers an efficient solution for building fast and compact cache stores.","/posts/building-a-fast-and-compact-sqlite-cache-store",[101,80,102,59],"bun","sqlite",{"_49":104},"\n

When working on applications, caching is crucial for enhancing performance by reducing the need for repeated database fetches. Among the various SQLite libraries available, Bun’s native integration is optimized for performance and takes advantage of its non-blocking I/O capabilities.

\n\n\n\n

\n
The bun:sqlite module is roughly 3-6x faster than better-sqlite3 and 8-9x faster than deno.land/x/sqlite for read queries. Each driver was benchmarked against the Northwind Traders dataset. View and run the benchmark source. Source
\n

\n\n\n\n

In addition to efficient caching, serializing JavaScript objects can be slow when using JSON. This is why it makes sense to opt for efficient binary encoding alternatives like Msgpackr or CBOR. These formats are faster to parse, support complex data types, require less CPU usage, and store data more compactly, further enhancing overall application performance.

\n\n\n\n

\n\n\n\n

With this in mind, let’s explore how to build a cache manager using bun:sqlite along with efficient binary encoding.

\n\n\n\n

A cache-manager Store

\n\n\n\n

cache-manager provides a straightforward and intuitive API for caching, abstracting away the complexity of managing different cache stores and their configurations. With support for multiple stores, built-in expiration and TTL management, and robust error handling and fallback mechanisms, it ensures data integrity and freshness.

\n\n\n\n

Additionally, cache-manager is highly customizable and extensible, allowing you to create custom cache stores tailored to your needs. This flexibility means you can set up and use caching with minimal code, allowing you to focus on your application’s core logic.

\n\n\n\n

The required interface to fulfill as a cache-manager store is as follows:

\n\n\n\n

export type Store = {\n    get<T>(key: string): Promise<T | undefined>;\n    set<T>(key: string, data: T, ttl?: Milliseconds): Promise<void>;\n    del(key: string): Promise<void>;\n    reset(): Promise<void>;\n    mset(arguments_: Array<[string, unknown]>, ttl?: Milliseconds): Promise<void>;\n    mget(...arguments_: string[]): Promise<unknown[]>;\n    mdel(...arguments_: string[]): Promise<void>;\n    keys(pattern?: string): Promise<string[]>;\n    ttl(key: string): Promise<number>;\n};

\n\n\n\n

See more here.

\n\n\n\n

The Queries

\n\n\n\n

Configuring SQLite for Optimal Performance

\n\n\n\n

PRAGMA main.synchronous = NORMAL;: Ensures that SQLite writes are fast while still maintaining a reasonable level of data safety. It does not guarantee as much durability as FULL, but it is sufficient for many use cases.
PRAGMA main.journal_mode = WAL2;: Improves concurrency by allowing readers to access the database while a write operation is ongoing.
PRAGMA main.auto_vacuum = INCREMENTAL;: Allows SQLite to reclaim unused space incrementally, rather than all at once.

\n\n\n\n

1. Creating the Cache Table

\n\n\n\n

CREATE TABLE IF NOT EXISTS {table} (\n    key TEXT PRIMARY KEY, \n    val BLOB, \n    created_at INTEGER, \n    expire_at INTEGER\n);\nCREATE INDEX IF NOT EXISTS index_expire_{table} ON {table}(expire_at);

\n\n\n\n

The val column stores the cached value in a binary large object (BLOB) format, allowing it to handle various data types depending on the chosen serializer.

\n\n\n\n

2. Inserting or Updating Cache Entries

\n\n\n\n

INSERT OR REPLACE INTO ${name}(key, val, created_at, expire_at) VALUES (?, ?, ?, ?)

\n\n\n\n

expire_at is pre-calculated based on the ttl value in milliseconds, and val is pre-checked by a function isCacheable: (value: unknown) => boolean;.

\n\n\n\n

3. Retrieving Cache Entries

\n\n\n\n

SELECT * FROM ${name} WHERE key = ? AND expire_at > ? LIMIT 1

\n\n\n\n

Returns one record that has not expired.

\n\n\n\n

Auto Purge and Batch Operations

\n\n\n\n

1. purgeExpired

\n\n\n\n

const purgeExpired = async () => {\n  const now = Date.now();\n  if (now - lastPurgeTime >= 60 * 60 * 1000) {\n    const statement = db.prepare(`DELETE FROM ${name} WHERE expire_at < ?`);\n    statement.run(now);\n    lastPurgeTime = now;\n  }\n};

\n\n\n\n

Keeps the cache clean and efficient by regularly removing stale entries, ensuring that the cache does not grow indefinitely and affect performance.

\n\n\n\n

2. Batch Set (mset)

\n\n\n\n

const mset = async (pairs: [string, unknown][], ttl?: number) => {\n  const ttlValue = ttl !== undefined ? ttl * 1000 : defaultTtl;\n  if (ttlValue < 0) {\n    return;\n  }\n  const expireAt = Date.now() + ttlValue;\n\n  const stmt = `INSERT OR REPLACE INTO ${name} (key, val, created_at, expire_at) VALUES ${pairs.map(() => '(?, ?, ?, ?)').join(', ')}`;\n  const bindings = pairs.flatMap(([key, value]) => {\n    if (!isCacheable(value)) {\n      throw new NoCacheableError(`\"${value}\" is not a cacheable value`);\n    }\n    return [key, serializerAdapter.serialize(value), Date.now(), expireAt];\n  });\n\n  const statement = db.prepare(stmt);\n  statement.run(...bindings);\n};

\n\n\n\n

Improves efficiency by reducing the number of individual database operations. In the same way mget executes one single query to returns valid records with the query:

\n\n\n\n

SELECT * FROM ${name} WHERE key IN (${placeholders}) AND expire_at > ?

\n\n\n\n

Conclusion

\n\n\n\n

Bun’s SQLite implementation combined with efficient binary encoding formats like Msgpackr provides a powerful solution for building fast and compact cache stores.

\n\n\n\n

For the complete source code and implementation details, visitt: https://github.com/sjdonado/cache-manager-bun-sqlite3

\n\n\n\n

You can also find the npm package cache-manager-bun-sqlite3.

\n\n\n\n

Happy hacking!

\n\n\n\n

\n",{"_35":106,"_37":107,"_39":108,"_41":109,"_8":110,"_44":111,"_47":113},12,"Validated forms with useFetcher in Remix","7/10/2024","Build a custom 'useFetcherForm' hook to easily handle fetcher requests.","/posts/validated-forms-with-usefetcher-in-remix",[80,112,58,59],"react",{"_49":114},"\n

Interacting with the server without using window.navigation significantly improves the user experience. E.g: Login forms within a dialog box or modal, optimistic UI forms or submitting multiple forms within a complex view.

\n\n\n\n

If you are not familiar with Remix or the useFetcher hook, please refer to:

\n\n\n\n

The FetcherForm Provider

\n\n\n\n

The Remix fetcher object contains three primary attributes: fetcher.state, fetcher.data and the method fetcher.submit. To interact with them we will use React.useEffect.

\n\n\n\n

This post will show how to create a provider that manages the state of a fetcher submitted form, along with a minimal custom hook:

\n\n\n\n

const { onChange, submitForm, isSubmitted, error } = useFetcherForm();

\n\n\n\n

Let’s break down the FetcherFormProvider props one by one.

\n\n\n\n

1) onChange

\n\n\n\n

External inputs will notify through this method to change the internal state of the provider, it receives a FormData argument:

\n\n\n\n

export default function FetcherFormProvider({\n  action,\n  method,\n  children,\n}: {\n  action: string;\n  method: SubmitOptions['method'];\n  children: React.ReactNode;\n}) {\n  const [formData, setFormData] = useState<FormData>();\n  [...]\n  return (\n    <FetcherFormContext.Provider\n      value={[\n        (formData: FormData) => {\n          setFormData(formData);\n        },\n        [...]\n        ]}\n    >\n      {children}\n    </FetcherFormContext.Provider>\n  );\n}

\n\n\n\n

2) submitForm

\n\n\n\n

Since the formData was already captured by the onChange method, the request can be send by calling fetcher.submit:

\n\n\n\n

export default function FetcherFormProvider({\n  action,\n  method,\n  children,\n}: {\n  action: string;\n  method: SubmitOptions['method'];\n  children: React.ReactNode;\n}) {\n  const fetcher = useFetcher();\n  const [formData, setFormData] = useState<FormData>();\n\n  [...]\n  return (\n    <FetcherFormContext.Provider\n      value={[\n        (formData: FormData) => {\n          setFormData(formData);\n        },\n        () => {\n          if (formData) {\n            fetcher.submit(formData, {\n              method,\n              action,\n            });\n          }\n        },\n        [...]\n      ]}\n    >\n      {children}\n    </FetcherFormContext.Provider>\n  );\n}

\n\n\n\n

3) isSubmitted

\n\n\n\n

Submitting the form is an asynchronous operation. There is a separate variable to listen to the form submitted event: isSubmitted. This is helpful in the following cases:

\n\n\n\n

To close the Dialog or Modal when the form is successfully submitted.
To check from outside that the form was submitted and/or the request returned an OK status.

\n\n\n\n

export default function FetcherFormProvider({\n  action,\n  method,\n  children,\n}: {\n  action: string;\n  method: SubmitOptions['method'];\n  children: React.ReactNode;\n}) {\n  const fetcher = useFetcher();\n\n+ const [isSubmitted, setIsSubmitted] = useState(false);\n  [...]\n  const [formData, setFormData] = useState<FormData>();\n\n  useEffect(() => {\n    const response = fetcher.data as { error: string } | undefined;\n\n+   if (isSubmitted || error) return;\n\n    if (fetcher.state === 'loading' && response) {\n      [...]\n+     setIsSubmitted(true);\n    }\n  }, [fetcher, action, formData, isSubmitted, error]);\n\n  return (\n    <FetcherFormContext.Provider\n      value={[\n        (formData: FormData) => {\n          setFormData(formData);\n        },\n        () => {\n          if (formData) {\n            fetcher.submit(formData, {\n              method,\n              action,\n            });\n          }\n        },\n+       isSubmitted,\n        [...]\n      ]}\n    >\n      {children}\n    </FetcherFormContext.Provider>\n  );\n}

\n\n\n\n

4) error

\n\n\n\n

One drawback of using Remix useFetcher is the lack of a straightforward error handling method. There are proposals in progress to provide a more streamlined error handling approach:

\n\n\n\n

https://github.com/remix-run/remix/discussions/4645
https://github.com/remix-run/react-router/discussions/10013

\n\n\n\n

As a workaround we can rely on fetcher.state to check if the request is complete and get the message with fetcher.data by defining a common structure between the client and server actions.

\n\n\n\n

The highlights

\n\n\n\n

const response = fetcher.data as { error: string } | undefined; -> defines the JSON response structure to be received form the server return json({ error: new Error() }, { status: 400 } );
if (fetcher.state === 'loading' && response) { -> checks if the request is completed and a response is available.

\n\n\n\n

export default function FetcherFormProvider({\n  action,\n  method,\n  children,\n}: {\n  action: string;\n  method: SubmitOptions['method'];\n  children: React.ReactNode;\n}) {\n  const fetcher = useFetcher();\n\n  const [isSubmitted, setIsSubmitted] = useState(false);\n+ const [error, setError] = useState<string>();\n\n  const [formData, setFormData] = useState<FormData>();\n\n  useEffect(() => {\n    const response = fetcher.data as { error: string } | undefined;\n\n    if (isSubmitted || error) return;\n\n    if (fetcher.state === 'loading' && response) {\n+     if (response.error) {\n+       setError(response.error);\n+       return;\n+     }\n      setIsSubmitted(true);\n    }\n  }, [fetcher, action, formData, isSubmitted, error]);\n\n  return (\n    <FetcherFormContext.Provider\n      value={[\n        (formData: FormData) => {\n          setFormData(formData);\n+         setError(undefined);\n        },\n        () => {\n          if (formData) {\n            fetcher.submit(formData, {\n              method,\n              action,\n            });\n          }\n        },\n        isSubmitted,\n+       error,\n      ]}\n    >\n      {children}\n    </FetcherFormContext.Provider>\n  );\n}

\n\n\n\n

Optionally, the error state management can be sent as a callback. This can be received in the onSubmit function and registered as state. For example, using a registeredCallback:

\n\n\n\n

- if (response.error) {\n-  setError(response.error);\n-  return;\n- }\n+ registeredCallback?.(response.error);

\n\n\n\n

The useFetcherForm hook

\n\n\n\n

The values sent to FetcherFormContext.Provider are defined in the FetcherFormContext

\n\n\n\n

const FetcherFormContext = createContext<\n  [(formData: FormData) => void, (callback?: () => void) => void, boolean, string?]\n>([() => null, () => null, false]);

\n\n\n\n

Then the hook exposes them in an object structure:

\n\n\n\n

export const useFetcherForm = () => {\n  const [onChange, submitForm, isSubmitted, error] = useContext(FetcherFormContext);\n  return {\n    onChange,\n    submitForm,\n    isSubmitted,\n    error,\n  };\n};

\n\n\n\n

An example of how it can be defined:

\n\n\n\n

export function AssignmentUpdateStatusDialogButton({\n  assignmentId,\n  status,\n}: {\n  assignmentId: string;\n  status: AssignmentStatus;\n}) {\n  const [isAttached, setIsAttached] = useState(false);\n  const dialog = useRef<HTMLDialogElement>();\n\n  useEffect(() => {\n    if (isAttached) {\n      dialog.current?.showModal();\n    }\n  }, [isAttached, dialog]);\n\n  return (\n    <ClientOnly>\n      {() => (\n        <>\n          {isAttached &&\n            createPortal(\n              <FetcherFormProvider\n                action={`/assignments/${assignmentId}/status`}\n                method=\"post\"\n              >\n                <AssignmentUpdateStatusDialog\n                  ref={dialog}\n                  [...]\n                  setIsAttached={setIsAttached}\n                />\n              </FetcherFormProvider>,\n              document.body\n            )}\n          <button\n            type=\"button\"\n            className=\"cursor-pointer leading-none\"\n            onClick={() => setIsAttached(true)}\n          >\n            <AssignmentStatusBadge status={status} />\n          </button>\n        </>\n      )}\n    </ClientOnly>\n  );\n}

\n\n\n\n

Demo

\n\n\n\n

Real-world example available on Github: https://github.com/sjdonado/remix-dashboard/blob/da9445646392626cea065442f7758230b3d8d1fa/app/components/dialog/AssignmentUpdateStatusDialog.tsx#L32C56-L32C70

\n\n\n\n

\n",{"_35":116,"_37":117,"_39":118,"_41":119,"_8":120,"_44":121,"_47":123},22,"Fast, Intuitive, Smart: Restaurant search engine with Cloudflare AI","4/13/2024","Discover \"Cule filo\" - An AI-Powered Restaurant Finder.","/posts/fast-intuitive-smart-restaurant-search-engine-with-cloudflare-ai",[122,80,59],"ai",{"_49":124},"\n

This project was submitted to the Cloudflare AI Challenge.

\n\n\n\n

The Inspiration Behind “Cule filo”

\n\n\n\n

Imagine you’re craving your favorite meal—a local specialty or a unique dish you tried once and can’t forget. Now, imagine finding the best spot for that meal near you, simply by typing the dish name and your location. That’s where “Cule filo” comes in, our AI-powered restaurant search engine that guides you to the top 3 restaurants serving exactly what you’re in the mood for. Built with powerful AI models and designed for simplicity, this free app is your smart companion to uncover culinary gems in your area.

\n\n\n

\n\n\n

Our Development Journey

\n\n\n\n

Building “Cule filo” was an exhilarating journey that started with a brainstorm on how to leverage Cloudflare’s AI tools to bring real value to users. After exploring various ideas, we zeroed in on the concept of a restaurant search engine that uses AI to make dining discovery intuitive and exciting.

\n\n\n\n

To make it happen, we brought together multiple AI models, each serving a distinct role in shaping a smooth user experience:

\n\n\n\n

Suggestions & Thumbnails: We used the llama-2-13b-chat-awq model to suggest alternative restaurants when fewer than 3 results matched the search, ensuring users always find relevant options. The same model selects thumbnails for the results, making them visually appealing.
Summarized Descriptions: For quick overviews of each restaurant, we turned to the bart-large-cnn model, which condenses user reviews into concise descriptions that capture each place’s unique qualities.
Image-to-Text Descriptions: Lastly, we used the uform-gen2-qwen-500m model to analyze restaurant photos and generate text descriptions, giving users a sense of the restaurant ambiance before they even step inside.

\n\n\n\n

Throughout development, we prioritized creating an interface that feels both responsive and intuitive. Real-time search logs keep users engaged, and a search history feature allows easy access to previous explorations.

\n\n\n\n

AI Models in Action

\n\n\n\n

Our project is particularly innovative in its use of multiple AI models per task and the integration of three distinct AI task types, making it a standout submission in the challenge:

\n\n\n\n

Multiple Models per Task:

\n\n\n\n

The llama-2-13b-chat-awq model is used both for generating suggestions when the initial search yields limited results and for selecting suitable thumbnails.
The bart-large-cnn model generates summaries of user reviews, giving a clear snapshot of what to expect at each restaurant.
The uform-gen2-qwen-500m model converts restaurant images into textual descriptions, adding depth and context to the search results.

\n\n\n\n

Triple Task Types:

\n\n\n\n

Text Generation: The llama-2-13b-chat-awq model generates recommendations and captions.
Text Summarization: The bart-large-cnn model condenses user reviews into summaries.
Image-to-Text: The uform-gen2-qwen-500m model creates descriptions based on restaurant images.

\n\n\n\n

To wrap up

\n\n\n\n

This project was a true team effort, and I couldn’t have done it without the creativity and support of my awesome friends and teammates: gjhernandez and krthr. Building this together was a blast, and we’re proud of what we’ve accomplished.

\n\n\n\n

If you’re curious about the inner workings or just want to poke around the code, you’re more than welcome to check out the repository: https://github.com/sjdonado/culefilo.

\n\n\n\n

\n",{"_35":126,"_37":127,"_39":128,"_41":109,"_8":129,"_44":130,"_47":132},27,"HTMX with Bun: A Real World App","3/5/2024","/posts/htmx-with-bun-a-real-world-app",[101,131,80,59],"htmx",{"_49":133},"\n

Let’s build a real-world app with HTMX + Tailwind CSS + Bun. If you are already familiar with these tools, feel free to skip the context part.

\n\n\n\n

Context

\n\n\n\n

HTMX, HTMZ, HTMY…

\n\n\n\n

All we need to achieve decent smooth reactivity is already out there, built-in by the major browsers. As an example, here is one interesting front-end framework:

\n\n\n\n

<iframe\n  hidden\n  name=htmz onload=\"setTimeout(()=>document.querySelector(contentWindow.location.hash||null)?.replaceWith(...contentDocument.body.childNodes))\"\n></iframe>

\n\n\n\n

That’s it, just an iframe, try it out yourself on https://leanrada.com/htmz/.

\n\n\n\n

The philosophy behind this is the same as HTMX and the same one that drives this post: Don’t reinvent rendering, just use HTML.

\n\n\n\n

The blazingly fast JS server

\n\n\n\n

With HTMX in mind, the next step is to get up and running a a simple web server.

\n\n\n\n

Bun has gained a lot of popularity in the last few months. It is fast, and there is a framework called ElysiaJS that resembles Express.js and promises to be 22 times faster than Express.

\n\n\n\n

Additionally, Bun understands TypeScript and JSX out of the box, so we can structure our app with components, server-side render them, and rely on HTMX for client-side actions.

\n\n\n\n

Tailwind CSS
Unlike traditional CSS frameworks that come with predefined components, Tailwind CSS emphasizes atomic classes that represent individual CSS properties. The bundle size could be lighter in comparison to other tools, but the strength here, IMHO, is rapid prototyping.

\n\n\n\n

A Real World App

\n\n\n\n

\n\n\n\n

The problem: I’m an Apple Music user. Someone shares a Spotify link with me, and I don’t have a way to know what’s inside or listen to it.

\n\n\n\n

The solution: I Don’t Have Spotify scrapes the links (songs/albums/artists etc) from all your favorite streaming services, and it even provides a listening preview if you just want a quick look.

\n\n\n\n

The approach

\n\n\n\n

Given a Spotify link, we need to extract its metadata: Fetch request + HTML parsing.
Given the metadata, we need to search for the resource on each supported streaming service: Multiple API/HTML requests + parsing (adapters).
Given the results of (1) and (2), we need to render and return the data to the client.
Additionally, we list a JSON endpoint for a cool Raycast extension.

\n\n\n\n

There are two major design decisions:

\n\n\n\n

The folder structure / separation of concerns
Tooling + build process, how to live reload the JSX components?

\n\n\n\n

For the first one, since we have multiple streaming services, we’ll inevitably encounter shared logic between them. Drawing inspiration from the adapter pattern but without the boilerplate, I organized the adapters folder with pure functions. These functions all receive the same arguments: query: string and metadata: SpotifyMetadata, and they return a SpotifyContentLink.

\n\n\n\n

The adapters are called by the spotifySearch function in the search service, which is also responsible for caching and updating the statistics. Why caching? In short: these calls are quite time-consuming, and we can easily hit a rate limit if we send many of them at the same time.

\n\n\n\n

For the second aspect, running Bun is very straightforward: bun run --watch www/bin.ts. However, as a real-world app, we need some JavaScript on the client side, apart from sending AJAX requests. There is an audio player that has to be rendered on the client side (in order to append listeners to the DOM). Additionally, we would like to access the Clipboard API to improve the user experience when searching.

\n\n\n\n

To bundle and minify the required JavaScript, I used Vite with the rollup-plugin-copy plugin:

\n\n\n\n

import path from 'path';\n\nimport { defineConfig } from 'vite';\nimport copy from 'rollup-plugin-copy';\n\nexport default defineConfig({\n  plugins: [\n    copy({\n      targets: [\n        {\n          src: 'dist/*',\n          dest: 'public',\n          copyOnce: false,\n        },\n      ],\n      hook: 'writeBundle',\n    }),\n  ],\n  resolve: {\n    alias: {\n      '~/config/constants': path.resolve(__dirname, './src/config/constants.ts'),\n    },\n  },\n  build: {\n    outDir: './dist',\n    target: 'esnext',\n    rollupOptions: {\n      input: {\n        'assets/js/audio-preview': './src/views/js/audio-preview.js',\n        'assets/js/search-bar': './src/views/js/search-bar.js',\n        'assets/css/index': './src/views/css/index.css',\n      },\n      output: {\n        entryFileNames: '[name].min.js',\n        chunkFileNames: `[name].min.js`,\n        assetFileNames: `[name].min.[ext]`,\n      },\n    },\n  },\n});

\n\n\n\n

Showcase

\n\n\n\n

The search bar component (where HTMX shines)

\n\n\n\n

export default function SearchBar() {\n  return (\n    <>\n      <form\n        id=\"search-form\"\n        hx-post=\"/search\"\n        hx-target=\"#search-results\"\n        hx-swap=\"innerHTML\"\n        hx-indicator=\"#loading-indicator\"\n        hx-request='\\\"timeout\\\":24000'\n        class=\"flex w-full max-w-3xl items-center justify-center\"\n      >\n        <label for=\"song-link\" class=\"sr-only\">\n          Search\n        </label>\n        <input\n          type=\"text\"\n          id=\"song-link\"\n          name=\"spotifyLink\"\n          class=\"flex-1 rounded-lg border bg-white p-2.5 text-sm font-normal text-black placeholder:text-gray-400 lg:text-base\"\n          placeholder=\"https://open.spotify.com/track/7A8MwSsu9efJXP6xvZfRN3?si=d4f1e2eb324c43df\"\n          pattern={SPOTIFY_LINK_REGEX.source}\n        />\n        <button\n          type=\"submit\"\n          class=\"ml-2 rounded-lg border border-green-500 bg-green-500 p-2.5 text-sm font-medium text-white focus:outline-none focus:ring-1 focus:ring-white\"\n        >\n          <i class=\"fas fa-search p-1 text-black\" />\n          <span class=\"sr-only\">Search</span>\n        </button>\n      </form>\n      <div class=\"my-4\">\n        <div id=\"search-results\"></div>\n      </div>\n    </>\n  );\n}

\n\n\n\n

The web server

\n\n\n\n

import { Elysia } from 'elysia';\nimport { html } from '@elysiajs/html';\nimport { staticPlugin } from '@elysiajs/static';\n\nimport { logger } from './utils/logger';\n\nimport { apiRouter } from './routes/api';\nimport { pageRouter } from './routes/page';\n\nexport const app = new Elysia()\n  .use(html())\n  .use(\n    staticPlugin({\n      prefix: '',\n      headers: {\n        'Cache-Control': 'public, max-age=86400',\n      },\n    })\n  )\n  .on('beforeHandle', async ({ request }) => {\n    logger.info(\n      `${request.method} ${request.url} - ${request.headers.get('user-agent')}`\n    );\n  })\n  .use(apiRouter)\n  .use(pageRouter);

\n\n\n\n

Testing

\n\n\n\n

In our case, testing involves mocks and more mocks. They run on push changes to master, thanks to GitHub Actions:

\n\n\n\n

The setup relies on bun:test and AxiosMockAdapter. It is separated into integration and unit tests, and the requests are injected using two helpers:

\n\n\n\n

export const JSONRequest = (endpoint: string, body: object) => {\n  return new Request(endpoint, {\n    method: 'POST',\n    headers: {\n      'Content-Type': 'application/json',\n    },\n    body: JSON.stringify(body),\n  });\n};\n\nexport const formDataRequest = (endpoint: string, body: object) => {\n  const formData = new FormData();\n\n  Object.entries(body).forEach(([key, value]) => {\n    formData.append(key, value);\n  });\n\n  return new Request(endpoint, {\n    method: 'POST',\n    body: formData,\n  });\n};

\n\n\n\n

Incorporating e2e shouldn’t be complex. An example of running the dev server with Playwright:

\n\n\n\n

  webServer: {
    command: 'bun run dev',
    port: 3333,
    reuseExistingServer: !process.env.CI,
  }

\n\n\n\n

Wrapping up

\n\n\n\n

The source code is available on GitHub: https://github.com/sjdonado/idonthavespotify.
The app is up and running thanks to Dokku: https://idonthavespotify.donado.co.
The Raycast extension will be addressed in a future post, stay tuned: https://www.raycast.com/sjdonado/idonthavespotify.

\n",{"_35":135,"_37":136,"_39":137,"_41":138,"_8":139,"_44":140,"_47":143},33,"Tower of Hanoi in P5.js + WASM","5/8/2023","Implementation of the Tower of Hanoi problem using P5.js for animation and Rust compiled to WebAssembly (WASM).","/posts/tower-of-hanoi-in-p5-js-wasm",[80,141,59,142],"p5.js","wasm",{"_49":144},"\n

In 2018, a professor at the Uni asked me to build a series of computer science games; he was looking for a very visual way to show students how the solution algorithms of NP problems (like the Knapsack Problem, Tower of Hanoi, Tic-Tac-Toe, etc.) behave and grow with their inputs.

\n\n\n

Tower of Hanoi is a good exercise when students are getting started with recursion and vectors. The pegs are usually fixed at 3, so it is easy to define the base case, and the only variable is the number of disks. It is worth noting that there are other more efficient approaches, such as The Binary solution or the Gray-code solution, but this was a Complexity course and we wanted to show how exponential time can break your computer while keeping it as minimal as possible.

\n\n\n\n

Vanilla JS Implementation

\n\n\n

There was only one design requirement: to show the animated solutions of the problems in a web browser. At that time I was familiar with HTML5 but not with any modern UI frameworks. Frankly, I couldn’t imagine myself rendering objects from scratch in a plain HTML canvas. There had to be something out there that had already invented that wheel, and there was.

\n\n\n\n

I found P5.js, it is sort of the JS version of Processing and it has everything I was looking for, a complete API for rendering, painting images and text, calling DOM elements and much more.

\n\n\n\n

The main core of the animation system was the draw function provided by P5. It works as an infinite loop, so if we want to show the disks moving from x1, y1 to x2, y2, we have to update the current position x += speed y += speed where 0 < speed < 120. There are two types of motion, vertical and horizontal. An example of the former is:

\n\n\n\n

if (isMovingHorizontallly) {\n  if (animation.currentMove.start < animation.currentMove.end) {\n    refreshCanvas(p5);\n    drawDiskByCoordinates(\n      p5,\n      animation.currentDisk,\n      animation.currentMove.start,\n      draw.topMargin,\n    );\n\n    animation.currentMove.start += game.speed;\n\n    if (animation.currentMove.start >= animation.currentMove.end) {\n      animation.currentMove.start = animation.currentMove.end;\n      animation.currentMove.down = draw.topMargin;\n    }\n  }\n  \n  if (animation.currentMove.start > animation.currentMove.end) {\n    refreshCanvas(p5);\n    drawDiskByCoordinates(\n      p5,\n      animation.currentDisk,\n      animation.currentMove.start,\n      draw.topMargin,\n    );\n\n    animation.currentMove.start -= game.speed;\n\n    if (animation.currentMove.start <= animation.currentMove.end) {\n      animation.currentMove.start = animation.currentMove.end;\n      animation.currentMove.down = draw.topMargin;\n    }\n  }\n}

\n\n\n\n

Note that the refreshCanvas function refreshes the canvas and redraws the towers filled by the disks, but not the one being animated.

\n\n\n\n

You may wonder how the disks know where to move, do they move at the same time as the Hanoi calculations? That would be cool, but unfortunately they don’t. We are limited by the recursive algorithm, so we have to wait until all calculations are done (until the call stack is empty).

\n\n\n\n

Therefore, the draw function constantly checks if the array of moves [towerFrom:towerTo, ...] generated by recursiveHanoi is not empty. If it is, the animation starts.

\n\n\n\n

function recursiveHanoi(A, C, B, n){\n  if (n == 1) {\n    moves.push(A + \":\" + C);\n  } else {\n    recursiveHanoi(A, B, C, n - 1);\n    moves.push(A + \":\" + C);\n    recursiveHanoi(B, C, A, n - 1);\n  }\n}\nfunction generateHanoiArray(disksNumber) {\n  var moves = [];\n  recursiveHanoi(0, 2, 1, disksNumber);\n  return moves;\n};

\n\n\n\n

recursiveHanoi did the job, but it was very inefficient: it couldn’t handle more than 14 disks (16K steps). The root cause was not in the algorithm itself, but in its execution in the V8 interpreter. Anyway, the same recursive function in C++ could run perfectly for more than 20 disks (aprox 16M of steps). So, running compiled bytecode in the browser was clearly the next step.

\n\n\n\n

wasm-pack

\n\n\n\n

After four years, I found some time to pay that deb-tech (yes, quite a long time, eh). To make it fun I rewrote everything from scratch in SolidJS, which went smooth thanks to this amazing library p5js-wrapper. For WASM, C++ is still a good choice, but what about Rust? I did some research and found wasm-pack. A few lines in the cargo.toml file and we were ready to generate compiled + ready to import bytecode!

\n\n\n

\n\n\n

Let’s dive into the configuration:

\n\n\n\n

The crate-type is set to indicate that the crate will be compiled as a dynamic library: wasm-bindgen.
gloo-utils and serde are included separately to avoid circular dependencies (history).

\n\n\n\n

... \n\n[lib]\npath = \"logic/lib.rs\"\ncrate-type = [\"cdylib\"]\n\n[dependencies]\nwasm-bindgen = \"0.2\"\ngetrandom = { version = \"0.2.8\", features = [\"js\"] }\ngloo-utils = { version = \"0.1\", features = [\"serde\"] }\nserde = { version = \"1.0\", features = [\"derive\"] }

\n\n\n\n

Once the configuration is ready, we run wasm-pack build -d wasm --no-typescript to compile. There is no advantage in generating ts files because we won’t use those definitions since the get_moves function is gonna be called using a Web Worker.

\n\n\n\n

Then, to make the wasm folder accessible to the solid app, we need to update the vite.config.js file following the recommended setup from vite-plugin-wasm:

\n\n\n\n

import { defineConfig } from 'vite';\n\nimport solidPlugin from 'vite-plugin-solid';\n+ import wasm from 'vite-plugin-wasm';\n+ import topLevelAwait from 'vite-plugin-top-level-await';\n\nimport path from 'path';\n\nexport default defineConfig({\n  plugins: [\n    solidPlugin(),\n+   wasm(),\n+   topLevelAwait(),\n  ],\n  server: {\n    port: 3000,\n  },\n  build: {\n    target: 'esnext',\n  },\n  resolve: {\n    alias: {\n      '@src': path.resolve(__dirname, './src'),\n+     '@wasm': path.resolve(__dirname, './wasm'),\n    },\n  },\n+  worker: {\n+    plugins: [\n+      wasm(),\n+      topLevelAwait(),\n+    ],\n+  },\n});

\n\n\n\n

The compiled output is now accessible as follows:

\n\n\n\n

import { get_moves } from '@wasm/games';

\n\n\n\n

But wait a second, we haven’t talked about this function yet, and why do we need a Web Worker? Let’s move on to the final implementation.

\n\n\n\n

Hanoi Algorithm in Rust

\n\n\n\n

The tricky part of writing WASM code is how to interact with the outside world (I/O operations). Fortunately, our function is quite small, its conditions are mapped as follows:

\n\n\n\n

The function takes a Number -> i32 argument: The number of disks.
The function returns an array of strings serialised by the serde_json crate (which allows you to convert a Rust data structure that implements the serde::Serialize trait to a serde_json::Value type, in this case a JsValue).

\n\n\n\n

And that’s it, inside the public fn, we can use regular Rust types like Vect.

\n\n\n\n

\nuse gloo_utils::format::JsValueSerdeExt;\nuse wasm_bindgen::prelude::*;\n\n#[wasm_bindgen]\npub fn get_moves(n: i32) -> JsValue {\n    fn move_tower(n: i32, from: i32, to: i32, aux: i32, moves: &mut Vec<String>) {\n        if n == 1 {\n            moves.push(format!(\"{}:{}\", from, to));\n        } else {\n            move_tower(n - 1, from, aux, to, moves);\n            moves.push(format!(\"{}:{}\", from, to));\n            move_tower(n - 1, aux, to, from, moves);\n        }\n    }\n\n    let mut moves = Vec::new();\n    move_tower(n, 0, 2, 1, &mut moves);\n    JsValue::from_serde(&moves).unwrap()\n}

\n\n\n\n

Web Worker

\n\n\n\n

As mentioned above, the number of disks is exponentially related to the number of steps required in the solution. 20 disks are about 1M, but 24 are more than 16M (which is quite a lot 😰). In fact, the execution time of the recursive function exceeds the Doherty threshold (300ms). So, it becomes mandatory to execute it inside a non-blocking thread, for which we can use the Web Workers API.

\n\n\n\n

A new .js file must be created for the worker anywhere in the src folder.

\n\n\n\n

import { get_moves } from '@wasm/games';\n\nonmessage = (event) => {\n  const moves = get_moves(event.data.n);\n\n  postMessage(moves);\n};

\n\n\n\n

Then, since the worker instance has nothing to do with the solid component lifecycle, we can instance it outside.

\n\n\n\n

const wasmWorker = new Worker(new URL('../workers/hanoi.js', import.meta.url), {\n  type: 'module',\n});

\n\n\n\n

Finally, when the user clicks on the play button, the async function is called:

\n\n\n\n

const getHanoiMoves = async (n) => new Promise((resolve) => {\n  wasmWorker.onmessage = (event) => resolve(event.data);\n  wasmWorker.postMessage({ n });\n});

\n\n\n\n

In Summary

\n\n\n\n

NP problems are highly CPU-intensive, and wasm-pack makes it easier to reduce the execution time. In addition, wrapping these operations in a Web Worker improves the user experience.

\n\n\n\n

A picture is worth a thousand words: for 20 disks the Vanilla JS implementation takes more than 312000ms, while Rust does it in under 200ms (tested on a Macbook Air M2).

\n\n\n\n

Vanilla JS	WASM (Rust)
$\"20$	$\"20$

\n\n\n\n

Source code available on Github: https://github.com/sjdonado/cs-games.

\n\n\n\n

\n","root","routes/_main.posts._index","actionData","errors"]

Blog

Filter by tag:

Managing 10k+ Visitors Overnight with Bare Metal