<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="3.10.0">Jekyll</generator><link href="https://epicserve.com//feed.xml" rel="self" type="application/atom+xml" /><link href="https://epicserve.com//" rel="alternate" type="text/html" /><updated>2024-12-07T00:00:43+00:00</updated><id>https://epicserve.com//feed.xml</id><title type="html">Epicserve</title><subtitle>Web development with epic service!</subtitle><author><name>Brent O&apos;Connor</name></author><entry><title type="html">Debugging Randomly Failing Tests with Reproducible Random Seeds</title><link href="https://epicserve.com//django/2024/10/29/debug-random-data.html" rel="alternate" type="text/html" title="Debugging Randomly Failing Tests with Reproducible Random Seeds" /><published>2024-10-29T13:58:00+00:00</published><updated>2024-10-29T13:58:00+00:00</updated><id>https://epicserve.com//django/2024/10/29/debug-random-data</id><content type="html" xml:base="https://epicserve.com//django/2024/10/29/debug-random-data.html"><![CDATA[<p>There are two things I despise in this world: dealing with printers that don’t work and debugging randomly
failing tests.</p>

<p>While I can’t solve all printer woes today, I can share how we make debugging flaky tests easier at <a href="https://canopyteam.org/">Canopy</a>. We
frequently use <a href="https://github.com/model-bakers/model_bakery">Model Bakery</a> and <a href="https://pypi.org/project/Faker/">Faker</a>
to generate test data, this randomly generated data can sometimes cause unpredictable test failures. To tackle this
challenge, we’ve implemented a solution that makes <a href="https://docs.pytest.org/">pytest</a> print the random seed number used
in failing tests. This allows us to reproduce the same test conditions by reusing that seed number in subsequent
test runs.</p>

<p>When a test fails, we can examine whether the randomly generated data is the culprit by rerunning the test with the
same seed. If the test fails consistently with that seed, we’ve likely identified that the test’s randomized data
contributes to the failure. While this isn’t always the root cause, it’s an excellent starting point for investigation.
(I should probably write another post about dealing with stubborn printers and other common causes of flaky tests!)</p>

<p>Here’s the code we use to implement this functionality. First, add this to your BaseTest class:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">BaseTest</span><span class="p">(</span><span class="n">TestCase</span><span class="p">):</span>
    <span class="o">@</span><span class="n">pytest</span><span class="p">.</span><span class="n">fixture</span><span class="p">(</span><span class="n">autouse</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
    <span class="k">def</span> <span class="nf">seed_random</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">request</span><span class="p">):</span>
        <span class="c1"># Use seed from the command line if provided
</span>        <span class="n">seed</span> <span class="o">=</span> <span class="n">request</span><span class="p">.</span><span class="n">config</span><span class="p">.</span><span class="n">getoption</span><span class="p">(</span><span class="s">"--seed"</span><span class="p">)</span>
        <span class="k">if</span> <span class="n">seed</span> <span class="ow">is</span> <span class="bp">None</span><span class="p">:</span>
            <span class="n">pytest</span><span class="p">.</span><span class="n">seed</span> <span class="o">=</span> <span class="n">random</span><span class="p">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">999999</span><span class="p">)</span>
        <span class="k">else</span><span class="p">:</span>
            <span class="n">pytest</span><span class="p">.</span><span class="n">seed</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">seed</span><span class="p">)</span>
        <span class="n">random</span><span class="p">.</span><span class="n">seed</span><span class="p">(</span><span class="n">pytest</span><span class="p">.</span><span class="n">seed</span><span class="p">)</span>
        <span class="n">Faker</span><span class="p">.</span><span class="n">seed</span><span class="p">(</span><span class="n">pytest</span><span class="p">.</span><span class="n">seed</span><span class="p">)</span>
        <span class="k">print</span><span class="p">(</span><span class="sa">f</span><span class="s">"</span><span class="se">\n</span><span class="s">Running test with seed: </span><span class="si">{</span><span class="n">pytest</span><span class="p">.</span><span class="n">seed</span><span class="si">}</span><span class="s">"</span><span class="p">)</span>
</code></pre></div></div>

<p>If you’re using function-based tests instead of class-based tests, you’ll want to add this fixture to your
<code class="language-plaintext highlighter-rouge">conftest.py</code> file:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">@</span><span class="n">pytest</span><span class="p">.</span><span class="n">fixture</span><span class="p">(</span><span class="n">autouse</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">seed_random</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
    <span class="c1"># Same implementation as above
</span>    <span class="p">...</span>
</code></pre></div></div>

<p>Finally, add this to your <code class="language-plaintext highlighter-rouge">conftest.py</code> to display the seed number when a test fails:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">pytest</span>


<span class="n">pytest</span><span class="p">.</span><span class="n">seed</span> <span class="o">=</span> <span class="bp">None</span>


<span class="k">def</span> <span class="nf">pytest_runtest_makereport</span><span class="p">(</span><span class="n">item</span><span class="p">,</span> <span class="n">call</span><span class="p">):</span>
    <span class="k">if</span> <span class="n">call</span><span class="p">.</span><span class="n">when</span> <span class="o">==</span> <span class="s">"call"</span> <span class="ow">and</span> <span class="n">call</span><span class="p">.</span><span class="n">excinfo</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">None</span><span class="p">:</span>
        <span class="k">print</span><span class="p">(</span>
            <span class="sa">f</span><span class="s">"</span><span class="se">\n</span><span class="si">{</span><span class="s">'*'</span> <span class="o">*</span> <span class="mi">22</span><span class="si">}</span><span class="se">\n</span><span class="s">Extra Failure Context:</span><span class="se">\n</span><span class="s">* Seed number used: </span><span class="si">{</span><span class="n">pytest</span><span class="p">.</span><span class="n">seed</span><span class="si">}</span><span class="s">. Use `pytest --seed </span><span class="si">{</span><span class="n">pytest</span><span class="p">.</span><span class="n">seed</span><span class="si">}</span><span class="s">` "</span>
            <span class="sa">f</span><span class="s">"to run the tests again with the same seed number.</span><span class="se">\n</span><span class="s">"</span>
        <span class="p">)</span>
</code></pre></div></div>

<p>Then you can run your tests with the <code class="language-plaintext highlighter-rouge">--seed</code> flag to reproduce the same random data and investigate the issue further
(e.g., <code class="language-plaintext highlighter-rouge">pytest --seed 34512</code>).</p>

<p>P.S. We have some amazing engineers at Canopy, and <a href="https://www.linkedin.com/in/levi-mann-8b34a6ab/">Levi Mann</a> is the
engineer at Canopy who gave us the idea. I’m just the one who used AI to help write it! 😂</p>]]></content><author><name>Brent O&apos;Connor</name></author><category term="django" /><summary type="html"><![CDATA[There are two things I despise in this world: dealing with printers that don’t work and debugging randomly failing tests.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://epicserve.com//assets/images/debug-random-data/debug-random-data-header.webp" /><media:content medium="image" url="https://epicserve.com//assets/images/debug-random-data/debug-random-data-header.webp" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Improving the New Django Developer Experience</title><link href="https://epicserve.com//django/2024/10/24/improving-the-new-django-developer-experience.html" rel="alternate" type="text/html" title="Improving the New Django Developer Experience" /><published>2024-10-24T21:26:00+00:00</published><updated>2024-10-24T21:26:00+00:00</updated><id>https://epicserve.com//django/2024/10/24/improving-the-new-django-developer-experience</id><content type="html" xml:base="https://epicserve.com//django/2024/10/24/improving-the-new-django-developer-experience.html"><![CDATA[<p>At DjangoCon 2024, I gave a lightning talk about improving the experience for new Django developers. While Django is an incredibly powerful framework, there’s one aspect of the initial developer experience that I believe we can significantly enhance. Let’s explore the current onboarding experience and how we might make it more intuitive for newcomers.</p>

<h2 id="the-current-django-project-setup">The Current Django Project Setup</h2>

<p>If you look at the official Django documentation, the tutorial instructs you to create a new project like this:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">mkdir </span>djangotutorial
<span class="nv">$ </span>django-admin startproject mysite djangotutorial
</code></pre></div></div>

<p>This creates the following directory structure:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">.</span>
├── manage.py
└── mysite
    ├── __init__.py
    ├── asgi.py
    ├── settings.py
    ├── urls.py
    └── wsgi.py
</code></pre></div></div>

<p>As someone who’s both learned and taught Django, I’ve noticed this structure raises several questions for newcomers:</p>

<ul>
  <li>Why create a <code class="language-plaintext highlighter-rouge">djangotutorial</code> directory only to have another directory called <code class="language-plaintext highlighter-rouge">mysite</code> inside it?</li>
  <li>What should the <code class="language-plaintext highlighter-rouge">mysite</code> directory be named in a real-world project?</li>
  <li>Why are configuration files housed in a project-named directory?</li>
  <li>How does this impact project maintainability, especially when version control and project renaming come into play?</li>
</ul>

<h2 id="learning-from-other-frameworks">Learning from Other Frameworks</h2>

<p>To provide context, let’s look at how other modern web frameworks handle project initialization. Laravel, for instance, offers a notably streamlined experience:</p>

<p><img alt="Laravel New Project Bash Session" src="/assets/images/better-django-experience/laravel-demo.gif" width="100%" /></p>

<p>Laravel places all configuration files in a dedicated <code class="language-plaintext highlighter-rouge">config</code> directory:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">.</span>
...
├── config
│   ├── app.php
│   ├── auth.php
│   ├── cache.php
│   ├── database.php
│   ├── filesystems.php
│   ├── logging.php
│   ├── mail.php
│   ├── queue.php
│   ├── services.php
│   └── session.php
...
</code></pre></div></div>

<p>Compare this to the current Django experience:</p>

<p><img alt="Django Start Project Bash Session" src="/assets/images/better-django-experience/django.gif" width="100%" /></p>

<p>The Django CLI experience has some rough edges - error messages aren’t consistently formatted, and simple issues like hyphens in project names result in errors rather than automatic conversion to underscores. The resulting structure still uses the project name for configuration files:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">.</span>
├── example_project
│   ├── __init__.py
│   ├── asgi.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
└── manage.py
</code></pre></div></div>

<h2 id="a-path-forward-dj-beat-drop">A Path Forward: DJ Beat Drop</h2>

<p>While changing Django’s core project templates might be challenging (as evidenced by <a href="https://github.com/django/django/pull/15609">this PR</a> that’s been open for two years), we can take a more pragmatic approach. I’ve created <a href="https://github.com/epicserve/dj-beat-drop">dj-beat-drop</a>, a modern Django project initializer that brings the best practices from other frameworks to the Django ecosystem.</p>

<p>DJ Beat Drop offers:</p>
<ul>
  <li>A more intuitive project structure</li>
  <li>It uses the official Django project template, but puts all configuration files in a <code class="language-plaintext highlighter-rouge">config</code> directory</li>
  <li>Built-in support for environment variables via <code class="language-plaintext highlighter-rouge">.env</code> files</li>
  <li>Integration with modern Python tooling like <code class="language-plaintext highlighter-rouge">uv</code></li>
  <li>A smoother, more user-friendly CLI experience</li>
</ul>

<p>Here’s what creating a new Django project with DJ Beat Drop looks like:</p>

<p><img alt="DJ Beat Drop New Project Bash Session" src="/assets/images/better-django-experience/beatdrop.gif" width="100%" /></p>

<p>With the defaults the resulting structure looks like this:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">.</span>
├── README.md
├── config
│   ├── __init__.py
│   ├── asgi.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
├── db.sqlite3
├── manage.py
├── pyproject.toml
└── uv.lock
</code></pre></div></div>

<p>If you say no to the defaults, then the resulting structure looks like this:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">.</span>
├── config
│   ├── __init__.py
│   ├── asgi.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
└── manage.py
</code></pre></div></div>

<h2 id="join-the-movement">Join the Movement</h2>

<p>If you’re interested in improving the Django developer experience, I invite you to try DJ Beat Drop and share your feedback. With community support, this could become the recommended way to start new Django projects, making the framework even more accessible to newcomers while maintaining the power and flexibility that Django developers love.</p>

<p>Want to help? Star the <a href="https://github.com/epicserve/dj-beat-drop">repository on GitHub</a> and share it with your fellow Djangonauts. Together, we can make the Django ecosystem even better for the next generation of developers.</p>]]></content><author><name>Brent O&apos;Connor</name></author><category term="django" /><summary type="html"><![CDATA[At DjangoCon 2024, I gave a lightning talk about improving the experience for new Django developers. While Django is an incredibly powerful framework, there’s one aspect of the initial developer experience that I believe we can significantly enhance. Let’s explore the current onboarding experience and how we might make it more intuitive for newcomers.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://epicserve.com//assets/images/better-django-experience/improving-the-new-django-developer-experience-header-image.webp" /><media:content medium="image" url="https://epicserve.com//assets/images/better-django-experience/improving-the-new-django-developer-experience-header-image.webp" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Node Permission Bug with Docker Compose</title><link href="https://epicserve.com//django/node/docker/2023/08/30/node-permission-bug.html" rel="alternate" type="text/html" title="Node Permission Bug with Docker Compose" /><published>2023-08-30T14:29:00+00:00</published><updated>2023-08-30T14:29:00+00:00</updated><id>https://epicserve.com//django/node/docker/2023/08/30/node-permission-bug</id><content type="html" xml:base="https://epicserve.com//django/node/docker/2023/08/30/node-permission-bug.html"><![CDATA[<p>When running Docker Compose for our project, which has a service for serving Python processes and another service for running Node processes, we started getting random permission errors like the following.</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>node    | Error: EACCES: permission denied, mkdir '/code/node_modules/.vite/deps_temp'
node    |     at Object.mkdirSync (node:fs:1349:3)
node    |     at runOptimizeDeps (file:///code/node_modules/vite/dist/node/chunks/dep-ca21228b.js:42881:14)
node    |     at Timeout._onTimeout (file:///code/node_modules/vite/dist/node/chunks/dep-ca21228b.js:42290:54)
node    |     at processTicksAndRejections (node:internal/process/task_queues:96:5) {
node    |   errno: -13,
node    |   syscall: 'mkdir',
node    |   code: 'EACCES',
node    |   path: '/code/node_modules/.vite/deps_temp'
node    | }
</code></pre></div></div>

<p>And:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>npm WARN logfile Error: EACCES: permission denied, scandir '/root/.npm/_logs'
npm WARN logfile  error cleaning log files [Error: EACCES: permission denied, scandir '/root/.npm/_logs'] {
npm WARN logfile   errno: -13,
npm WARN logfile   code: 'EACCES',
npm WARN logfile   syscall: 'scandir',
npm WARN logfile   path: '/root/.npm/_logs'
npm WARN logfile }
npm ERR! code EACCESS
npm ERR! syscall mkdir
npm ERR! path /root/.npm/_cacache/tmp
npm ERR! errno -13
npm ERR! 
</code></pre></div></div>

<p>These errors didn’t make much sense to us because the errors were random, and the user that the node service was running as should have been root. What we discovered was that we were running into an issue where any scripts run with <code class="language-plaintext highlighter-rouge">npm run</code> were switching to running the process as the owner of a path based on the owner of its nearest existing parent by using <a href="https://www.npmjs.com/package/infer-owner">infer-owner</a>. This behavior has now been <a href="https://github.com/npm/promise-spawn/pull/40">removed</a> from NPM’s dependency promise-spawn as of v5.0.0 and NPM version <a href="https://github.com/npm/cli/blob/latest/CHANGELOG.md#901-2022-10-26">v9.0.1</a>. The owner of our project’s root folder was getting changed to a non-privileged user by our Django/Python process because we had set up that service to run as a generic app user that we created for security reasons. We’re still not 100% sure which Python script/process was changing to the root folder to be owned by the generic app user. But this is why <code class="language-plaintext highlighter-rouge">npm run</code> switched to running by the user ID of the generic app user since the generic app user didn’t exist in the node container.</p>

<p>We were also puzzled by why the file and directory owner and group changes persisted for our project directories. Through trial and error, we discovered that Docker stores owner and group changes using Mac’s extended attributes on a Mac host. When we looked at the project directories’ extended attributes by running <code class="language-plaintext highlighter-rouge">xattr .</code> on our Macs, we discovered there was an attribute named <code class="language-plaintext highlighter-rouge">com.docker.grpcfuse.ownership</code> when we printed the value of the attribute using <code class="language-plaintext highlighter-rouge">xattr -p com.docker.grpcfuse.ownership .</code> the result <code class="language-plaintext highlighter-rouge">{"UID":33,"GID":33,"mode":10000}</code> showed that was where Docker was storing the owner changes. After deleting the attribute with <code class="language-plaintext highlighter-rouge">xattr -d com.docker.grpcfuse.ownership .</code>, the directory’s owner returned to being root.</p>

<p>Our current solution is to ensure we are running our Python/Django service as the same user as our Node service is using. Right now, we’re just using root for both services since that’s straightforward and since this is only for local development. However, upgrading NPM could also fix this or switch to creating an app user with the same ID in both services.</p>

<p>My hope with this blog post is that others might find it and save some time in debugging and figuring out a solution. Our team learned a lot about Docker and Node in the process.</p>]]></content><author><name>Brent O&apos;Connor</name></author><category term="django" /><category term="node" /><category term="docker" /><summary type="html"><![CDATA[When running Docker Compose for our project, which has a service for serving Python processes and another service for running Node processes, we started getting random permission errors like the following.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://epicserve.com//assets/images/epicserve-default-social-share-logo.png" /><media:content medium="image" url="https://epicserve.com//assets/images/epicserve-default-social-share-logo.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Use Fly.io with the Django Base Site</title><link href="https://epicserve.com//django/2022/12/30/using-flyio-with-the-django-base-site.html" rel="alternate" type="text/html" title="Use Fly.io with the Django Base Site" /><published>2022-12-30T12:17:00+00:00</published><updated>2022-12-30T12:17:00+00:00</updated><id>https://epicserve.com//django/2022/12/30/using-flyio-with-the-django-base-site</id><content type="html" xml:base="https://epicserve.com//django/2022/12/30/using-flyio-with-the-django-base-site.html"><![CDATA[<p>Since Heroku no longer has a free plan, I wanted to try out <a href="https://fly.io/">fly.io</a> that I’ve heard about as an
alternative to Heroku. For me there is no better way to try it than to use my
<a href="https://github.com/epicserve/django-base-site">Django Base Site</a>. In the past, I made sure that my starter template
project worked with Heroku, so it makes sense that I should try it out with a new and up and coming PaaS.</p>

<p>As of the writing of this blog post, these are the steps I used to deploy the Django Base Site to Fly.io.</p>

<ol>
  <li><a href="https://fly.io/docs/hands-on/install-flyctl/">Install flyctl</a></li>
  <li>Run <code class="language-plaintext highlighter-rouge">fly launch</code> and answer all the input prompts. Make sure you respond with <code class="language-plaintext highlighter-rouge">y</code> when it asks to set up Postgres and
Redis. It’s important to note that if you choose that you would like it to deploy now it will fail, so choose no.
Fly.io’s default configuration is to use Heroku buildpacks, which should work with the Django Base Site, but for
whatever reason I wasn’t able to get it work by using the Heroku buildpack for builder. If interested have a look at
the steps I tried below.
    <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>➜ fly launch
Creating app in /Users/brento/Sites/personal/django-base-site
Scanning source code
Detected a NodeJS app
Using the following build configuration:
        Builder: heroku/buildpacks:20
? Choose an app name (leave blank to generate one): django-base-site
automatically selected personal organization: Brent O'Connor
? Choose a region for deployment: Dallas, Texas (US) (dfw)
Created app django-base-site in organization personal
Admin URL: https://fly.io/apps/django-base-site
Hostname: django-base-site.fly.dev
Wrote config file fly.toml
? Would you like to set up a Postgresql database now? Yes
? Select configuration: Development - Single node, 1x shared CPU, 256MB RAM, 1GB disk
Creating postgres cluster in organization personal
Creating app...
Setting secrets on app django-base-site-db...
Provisioning 1 of 1 machines with image flyio/postgres:14.4
Waiting for machine to start...
Machine 3d8d3d7fe56989 is created
==&gt; Monitoring health checks
  Waiting for 3d8d3d7fe56989 to become healthy (started, 3/3)
   
Postgres cluster django-base-site-db created
  Username:    postgres
  Password:    &lt;redacted&gt;
  Hostname:    django-base-site-db.internal
  Proxy port:  5432
  Postgres port:  5433
  Connection string: postgres://postgres:&lt;redacted&gt;@django-base-site-db.internal:5432
   
Save your credentials in a secure place -- you won't be able to see them again!
   
Connect to postgres
Any app within the Brent O'Connor organization can connect to this Postgres using the following connection string:
   
Now that you've set up Postgres, here's what you need to understand: https://fly.io/docs/postgres/getting-started/what-you-should-know/
   
Postgres cluster django-base-site-db is now attached to django-base-site
The following secret was added to django-base-site:
  DATABASE_URL=postgres://django_base_site:tERgYWZ1jV2fHbJ@top2.nearest.of.django-base-site-db.internal:5432/django_base_site?sslmode=disable
Postgres cluster django-base-site-db is now attached to django-base-site
? Would you like to set up an Upstash Redis database now? Yes
? Select an Upstash Redis plan Free: 100 MB Max Data Size
input:3: createAddOn Validation failed: Name has already been taken
   
? Would you like to deploy now? No
</code></pre></div>    </div>
  </li>
  <li>Edit the <code class="language-plaintext highlighter-rouge">fly.toml</code> file the previous command created by updating the following sections to match below. Also
make sure you replace <code class="language-plaintext highlighter-rouge">&lt;app_name&gt;</code> with the name of the app that was created when you ran <code class="language-plaintext highlighter-rouge">fly launch</code>.
    <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>[build]
  dockerfile = "config/docker/Dockerfile"

[build.args]
  ENV_NAME = "prod"

[deploy]
  release_command = "python manage.py migrate --noinput"

[env]
  PORT = "8080"
  ALLOWED_HOSTS = "&lt;app_name&gt;.fly.dev"
  INTERNAL_IPS = "&lt;app_name&gt;.fly.dev"
  DB_SSL_REQUIRED = "off"
</code></pre></div>    </div>
  </li>
  <li>Set the <code class="language-plaintext highlighter-rouge">SECRET_KEY</code> as a secret environment variable:
    <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>fly secrets set SECRET_KEY=$(python -c "import random; print(''.join(random.SystemRandom().choice('abcdefghijklmnopqrstuvwxyz0123456789%^&amp;*(-_=+)') for i in range(50)))")
</code></pre></div>    </div>
  </li>
  <li>Run <code class="language-plaintext highlighter-rouge">fly deploy</code> to deploy your app to Fly.io.</li>
  <li>Run <code class="language-plaintext highlighter-rouge">fly ssh console</code> and then run <code class="language-plaintext highlighter-rouge">cd /srv/app &amp;&amp; ./manage.py migrate &amp;&amp; ./manage.py createsuperuser</code> to create your
user for signing in. Exit your ssh session.</li>
  <li>Run <code class="language-plaintext highlighter-rouge">fly open</code> to open the app in your browser. You won’t be able to login via /accounts/login/ until you validate
your email address. To do this, go to /admin/ and sign in. Then go to /admin/account/emailaddress/ and mark your
email address as primary and validated. Then you should be able to sign in with the standard sign-in view. If your
app was set up to send email, you wouldn’t have to validate your email address in the admin first because when you
sign in, the app will send you an email with a link to click on to validate your email address.</li>
  <li>Hurray, you’ve successfully deployed to Fly.io! 🎉</li>
</ol>

<h2 id="troubleshooting">Troubleshooting</h2>

<p>If you get a 500 error after you sign in or at any point with your running application and need to debug it, you can add
the following to your settings and then run <code class="language-plaintext highlighter-rouge">fly deploy</code>. Once the app deploys, you can trigger the 500
error again, and then when you run <code class="language-plaintext highlighter-rouge">fly logs</code>, you should be able to see a python traceback of the exception error. This
is a quick and easy solution for debugging, but a better solution for production environments is to set up something
like <a href="https://sentry.io/">Sentry</a>.</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
        },
    },
    'root': {
        'handlers': ['console'],
        'level': 'INFO',
    },
    'loggers': {
        'django': {
            'handlers': ['console'],
            'level': 'INFO',
            'propagate': False,
        },
    },
}
</code></pre></div></div>

<h2 id="steps-i-tried-to-get-flyio-working-with-heroku-buildpacks">Steps I tried to get Fly.io working with Heroku Buildpacks</h2>

<ol>
  <li>I added a <code class="language-plaintext highlighter-rouge">requirements.txt</code> file in the root of the project so the Heroku Buildpack would detect that the project is
a Python application. Inside the file I pointed to my production requirements
file <code class="language-plaintext highlighter-rouge">-r config/requirements/prod_lock.txt</code>.</li>
  <li>Then I ran <code class="language-plaintext highlighter-rouge">fly deploy</code>, which failed because it couldn’t install the requirements and quit with the following error.
    <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
    tomli from https://files.pythonhosted.org/packages/97/75/10a9ebee3fd790d20926a90a2547f0bf78f371b2f13aa822c759680ca7b9/tomli-2.0.1-py3-none-any.whl (from coverage[toml]==7.0.1-&gt;-r /workspace/config/requirements/prod_lock.txt (line 131))
</code></pre></div>    </div>
  </li>
  <li>Next, I tried pointing the requirements to the un-hashed version of my
requirements (<code class="language-plaintext highlighter-rouge">-r config/requirements/prod.in</code>) and then ran <code class="language-plaintext highlighter-rouge">fly deploy</code> again. This ultimately failed again, but
this time it failed after installing the requirements and failed because it wasn’t able to read the <code class="language-plaintext highlighter-rouge">SECRET_KEY</code>
environment variable.
    <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>      File "/workspace/config/settings/_base.py", line 28, in &lt;module&gt;
        SECRET_KEY = env("SECRET_KEY")
      File "/app/.heroku/python/lib/python3.10/site-packages/environs/__init__.py", line 116, in method
        raise EnvError('Environment variable "{}" not set'.format(proxied_key or parsed_key))
      environs.EnvError: Environment variable "SECRET_KEY" not set

!     Error while running '$ python manage.py collectstatic --noinput'.
      See traceback above for details.

      You may need to update application code to resolve this error.
      Or, you can disable collectstatic for this application:

      $ heroku config:set DISABLE_COLLECTSTATIC=1
</code></pre></div>    </div>
  </li>
  <li>Logically, I tried setting the <code class="language-plaintext highlighter-rouge">SECRET_KEY</code> using <code class="language-plaintext highlighter-rouge">fly secrets set SECRET_KEY=" &lt;redacted&gt;" </code> and then I tried
deploying again with <code class="language-plaintext highlighter-rouge">fly deploy</code>, but ultimately that failed again with the same error as the previous step. As
another troubleshooting step, I also tried removing the <code class="language-plaintext highlighter-rouge">SECRET_KEY</code> with <code class="language-plaintext highlighter-rouge">fly secrets unset SECRET_KEY</code> and then
adding it to the <code class="language-plaintext highlighter-rouge">[env]</code> section of the <code class="language-plaintext highlighter-rouge">fly.toml</code> file. This also failed with the same error when I ran
<code class="language-plaintext highlighter-rouge">fly deploy</code>. This is where I gave up and tried the approach above by using the Django Base Site Dockerfile.</li>
</ol>

<p>If anyone has any other ideas or can get the Django Base Site working with
fly.io <a href="mailto:brent@epicserve.com">please let me know</a>.</p>

<h2 id="conclusions">Conclusions</h2>

<p>It would be nice to figure out why Heroku buildpacks (the default builder) didn’t work for me at some point
because I had to create a pretty sophisticated Dockerfile to get Fly.io working. It makes me wonder if
other people using Fly.io for a Django project run into the same problems or if it’s just something that unique to my
Django Base Site.</p>

<p>If you want to deploy a hobby project to the world quickly then Fly.io seems like a good choice. Especially with its
free plan. One of the things I liked about Fly.io is that it has intelligent caching. If you haven’t made changes to
project requirement files, it deploys quickly (less than a minute), which is very nice!</p>]]></content><author><name>Brent O&apos;Connor</name></author><category term="django" /><summary type="html"><![CDATA[Since Heroku no longer has a free plan, I wanted to try out fly.io that I’ve heard about as an alternative to Heroku. For me there is no better way to try it than to use my Django Base Site. In the past, I made sure that my starter template project worked with Heroku, so it makes sense that I should try it out with a new and up and coming PaaS.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://epicserve.com//assets/images/epicserve-default-social-share-logo.png" /><media:content medium="image" url="https://epicserve.com//assets/images/epicserve-default-social-share-logo.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Fix GitHub Action Docker Compose Permission Errors</title><link href="https://epicserve.com//django/2022/12/26/github-action-permission-errors.html" rel="alternate" type="text/html" title="Fix GitHub Action Docker Compose Permission Errors" /><published>2022-12-26T16:34:00+00:00</published><updated>2022-12-26T16:34:00+00:00</updated><id>https://epicserve.com//django/2022/12/26/github-action-permission-errors</id><content type="html" xml:base="https://epicserve.com//django/2022/12/26/github-action-permission-errors.html"><![CDATA[<p>I was working on my <a href="https://github.com/epicserve/django-base-site">Django Base Site</a> project when I discovered a test
was failing in a GitHub Action when trying to run Django’s <code class="language-plaintext highlighter-rouge">collectstatic</code> manager command. It was failing in Python
with the error, <code class="language-plaintext highlighter-rouge">PermissionError: [Errno 13] Permission denied: '/srv/app/collected_static'</code>.</p>

<p>I’ll save you from all the boring details from my hours of debugging and trying to fix it. What ending up being the fix
was simply running my Docker Compose commands as root with the <code class="language-plaintext highlighter-rouge">-u root</code> argument.</p>

<p>Example:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker compose run <span class="nt">-u</span> root <span class="nt">--rm</span> <span class="nt">--no-deps</span> web ./manage.py collectstatic <span class="nt">--no-input</span>
</code></pre></div></div>

<p>The reason I had to do this is that my Docker image is built using the <code class="language-plaintext highlighter-rouge">USER app</code> instruction in the Dockerfile and
according to the GitHub documentation on the Dockerfile it <a href="https://docs.github.com/en/actions/creating-actions/dockerfile-support-for-github-actions#user">says</a>:</p>

<blockquote>
  <p>Docker actions must be run by the default Docker user (root). Do not use the USER instruction in your Dockerfile,
because you won’t be able to access the GITHUB_WORKSPACE. For more information, see “Using environment variables” and
USER reference in the Docker documentation.</p>
</blockquote>

<p>As a bonus, if you’re debugging GitHub Actions, a fantastic tool that can save you hours of frustration is to use the
<a href="https://github.com/mxschmitt/action-tmate">mxschmitt/action-tmate</a> action. Instead of making a change to your Github Action YAML file and then pushing the
change and waiting to see if it passes. You can use this action to create a live interactive shell for the container
your action is running in so that you can quickly test and try new fixes.</p>]]></content><author><name>Brent O&apos;Connor</name></author><category term="django" /><summary type="html"><![CDATA[I was working on my Django Base Site project when I discovered a test was failing in a GitHub Action when trying to run Django’s collectstatic manager command. It was failing in Python with the error, PermissionError: [Errno 13] Permission denied: '/srv/app/collected_static'.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://epicserve.com//assets/images/epicserve-default-social-share-logo.png" /><media:content medium="image" url="https://epicserve.com//assets/images/epicserve-default-social-share-logo.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Serialize Django Data for Javascript</title><link href="https://epicserve.com//django/2022/08/27/serialize-django-data-for-js.html" rel="alternate" type="text/html" title="Serialize Django Data for Javascript" /><published>2022-08-27T21:11:00+00:00</published><updated>2022-08-27T21:11:00+00:00</updated><id>https://epicserve.com//django/2022/08/27/serialize-django-data-for-js</id><content type="html" xml:base="https://epicserve.com//django/2022/08/27/serialize-django-data-for-js.html"><![CDATA[<p>Serializing Django view context data for use in Javascript is something I’ve
always thought should be extremely easy, but any time you have data that is more
then a simple data type like a string or an integer, it can get complicated
pretty quickly and can end up causing hard-to-solve bugs.</p>

<p>In most cases, if you have context data in your view like the following:</p>

<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">data</span> <span class="o">=</span> <span class="p">[</span>
    <span class="p">{</span>
        <span class="s">"mystr"</span><span class="p">:</span> <span class="s">"Foo"</span><span class="p">,</span>
        <span class="s">"myint"</span><span class="p">:</span> <span class="mi">123</span><span class="p">,</span>
    <span class="p">},</span>
<span class="p">]</span></code></pre></figure>

<p>Then all you have to do to make it work in a template is to use the <a href="https://docs.djangoproject.com/en/4.1/ref/templates/builtins/#std-templatefilter-safe">safe</a>
template filter and then it just works. Example:</p>

<figure class="highlight"><pre><code class="language-html" data-lang="html"><span class="nt">&lt;script&gt;</span>
  <span class="kd">function</span> <span class="nx">send_data_to_js_example</span><span class="p">(</span><span class="nx">data</span><span class="p">)</span> <span class="p">{</span>
    <span class="nx">console</span><span class="p">.</span><span class="nx">assert</span><span class="p">(</span><span class="nx">data</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">mystr</span> <span class="o">===</span> <span class="dl">"</span><span class="s2">Foo</span><span class="dl">"</span><span class="p">);</span>
    <span class="nx">console</span><span class="p">.</span><span class="nx">assert</span><span class="p">(</span><span class="nx">data</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">myint</span> <span class="o">===</span> <span class="mi">123</span><span class="p">);</span>
  <span class="p">}</span>
  <span class="nx">send_data_to_js_example</span><span class="p">({{</span> <span class="nx">data</span><span class="o">|</span><span class="nx">safe</span> <span class="p">}});</span>
<span class="nt">&lt;/script&gt;</span></code></pre></figure>

<p>But what if your data is more complex like the following example?</p>

<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">data</span> <span class="o">=</span> <span class="p">[</span>
    <span class="p">{</span>
        <span class="s">"mystr"</span><span class="p">:</span> <span class="s">"Foo"</span><span class="p">,</span>
        <span class="s">"myint"</span><span class="p">:</span> <span class="mi">123</span><span class="p">,</span>
        <span class="s">"myfloat"</span><span class="p">:</span> <span class="mf">123.45</span><span class="p">,</span>
        <span class="mi">456</span><span class="p">:</span> <span class="s">"abc"</span><span class="p">,</span>
        <span class="s">"truebool"</span><span class="p">:</span> <span class="bp">True</span><span class="p">,</span>
        <span class="s">"falsebool"</span><span class="p">:</span> <span class="bp">False</span><span class="p">,</span>
        <span class="s">"list"</span><span class="p">:</span> <span class="p">[</span><span class="s">"a"</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="s">"b"</span><span class="p">],</span>
        <span class="s">"user"</span><span class="p">:</span> <span class="p">{</span><span class="s">"id"</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span> <span class="s">"name"</span><span class="p">:</span> <span class="s">"Joe Example"</span><span class="p">},</span>
        <span class="s">"user_list"</span><span class="p">:</span> <span class="n">User</span><span class="p">.</span><span class="n">objects</span><span class="p">.</span><span class="nb">all</span><span class="p">(),</span>
        <span class="s">"date_time"</span><span class="p">:</span> <span class="n">now</span><span class="p">(),</span>
        <span class="s">"html"</span><span class="p">:</span> <span class="s">'&lt;p class="m-100 float-left random modifier p-100 spacer-5 john-b-good"&gt;My &lt;strong&gt;Paragraph&lt;/strong&gt;&lt;/p&gt;'</span><span class="p">,</span>
    <span class="p">},</span>
<span class="p">]</span></code></pre></figure>

<p>Using the <code class="language-plaintext highlighter-rouge">safe</code> template filter no longer works because your booleans aren’t
converted to lowercase for use in Javascript as well as some of the other data
types like the QuerySet, the Python datetime object, and your HTML string.</p>

<p>You might try using something like the <a href="https://docs.djangoproject.com/en/4.1/ref/templates/builtins/#escapejs">escapejs</a> hoping
to find some easy template filter that just works. However, I’ll save you the
trouble; it doesn’t work and the <a href="https://docs.djangoproject.com/en/4.1/ref/templates/builtins/#escapejs">escapejs</a> filter is only meant for
use on a single variable and not a list of dicts. You also might try using
<code class="language-plaintext highlighter-rouge">json.dumps()</code> with <a href="https://docs.djangoproject.com/en/4.1/topics/serialization/">Django’s serialize()</a> function, but as
I discovered this doesn’t work so well for most of my use cases because the
<code class="language-plaintext highlighter-rouge">serialize()</code> function doesn’t serialize the model instances in a flattened
structure and serializes the data in a more nested structure. Example:</p>

<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="p">[</span>
  <span class="p">{</span>
    <span class="s">"model"</span><span class="p">:</span> <span class="s">"accounts.user"</span><span class="p">,</span>
    <span class="s">"pk"</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
    <span class="s">"fields"</span><span class="p">:</span> <span class="p">{</span>
      <span class="s">"first_name"</span><span class="p">:</span> <span class="s">"Joe"</span><span class="p">,</span>
      <span class="s">"last_name"</span><span class="p">:</span> <span class="s">"Example"</span><span class="p">,</span>
      <span class="s">"email"</span><span class="p">:</span> <span class="s">"joe@examle.com"</span><span class="p">,</span>
      <span class="p">...</span>
    <span class="p">}</span>
  <span class="p">}</span>
<span class="p">]</span></code></pre></figure>

<h2 id="the-magic-solution">The Magic Solution</h2>

<p>So if you’re wanting a Django template filter that just works, you’ll have to
create your own. The following is the solution I came up with that just works
and extends Django’s <a href="https://docs.djangoproject.com/en/4.1/topics/serialization/#djangojsonencoder">DjangoJSONEncoder</a> encoder class which
adds the functionality to serialize QuerySets.</p>

<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="k">class</span> <span class="nc">DjangoModelJSONEncoder</span><span class="p">(</span><span class="n">DjangoJSONEncoder</span><span class="p">):</span>
    <span class="k">def</span> <span class="nf">default</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">o</span><span class="p">):</span>
        <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">o</span><span class="p">,</span> <span class="n">QuerySet</span><span class="p">)</span> <span class="ow">is</span> <span class="bp">True</span><span class="p">:</span>
            <span class="k">if</span> <span class="n">o</span><span class="p">.</span><span class="n">_iterable_class</span> <span class="ow">is</span> <span class="n">ModelIterable</span><span class="p">:</span>
                <span class="n">o</span> <span class="o">=</span> <span class="n">o</span><span class="p">.</span><span class="n">values</span><span class="p">()</span>
            <span class="k">return</span> <span class="nb">list</span><span class="p">(</span><span class="n">o</span><span class="p">)</span>
        <span class="k">return</span> <span class="nb">super</span><span class="p">().</span><span class="n">default</span><span class="p">(</span><span class="n">o</span><span class="p">)</span>


<span class="o">@</span><span class="n">register</span><span class="p">.</span><span class="nb">filter</span>
<span class="k">def</span> <span class="nf">to_json</span><span class="p">(</span><span class="n">value</span><span class="p">:</span> <span class="n">Any</span><span class="p">,</span> <span class="n">indent</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="bp">None</span><span class="p">):</span>
    <span class="k">return</span> <span class="n">mark_safe</span><span class="p">(</span><span class="n">json</span><span class="p">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">value</span><span class="p">,</span> <span class="n">cls</span><span class="o">=</span><span class="n">DjangoModelJSONEncoder</span><span class="p">,</span> <span class="n">indent</span><span class="o">=</span><span class="n">indent</span><span class="p">).</span><span class="n">translate</span><span class="p">({</span>
        <span class="nb">ord</span><span class="p">(</span><span class="s">"&gt;"</span><span class="p">):</span> <span class="s">"</span><span class="se">\\</span><span class="s">u003E"</span><span class="p">,</span>
        <span class="nb">ord</span><span class="p">(</span><span class="s">"&lt;"</span><span class="p">):</span> <span class="s">"</span><span class="se">\\</span><span class="s">u003C"</span><span class="p">,</span>
        <span class="nb">ord</span><span class="p">(</span><span class="s">"&amp;"</span><span class="p">):</span> <span class="s">"</span><span class="se">\\</span><span class="s">u0026"</span><span class="p">,</span>
    <span class="p">}))</span></code></pre></figure>

<p>With the previous <code class="language-plaintext highlighter-rouge">to_json</code> template filter created, then using it in a template
like the following just works!</p>

<figure class="highlight"><pre><code class="language-html" data-lang="html"><span class="nt">&lt;script&gt;</span>
  <span class="kd">function</span> <span class="nx">get_data_from_data_attribute</span><span class="p">()</span> <span class="p">{</span>
    <span class="nx">data</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">date_time</span> <span class="o">=</span> <span class="k">new</span> <span class="nb">Date</span><span class="p">(</span><span class="nx">data</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">date_time</span><span class="p">);</span>
    <span class="nx">console</span><span class="p">.</span><span class="nx">assert</span><span class="p">(</span><span class="nx">data</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">mystr</span> <span class="o">===</span> <span class="dl">"</span><span class="s2">Foo</span><span class="dl">"</span><span class="p">);</span>
    <span class="nx">console</span><span class="p">.</span><span class="nx">assert</span><span class="p">(</span><span class="nx">data</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">myint</span> <span class="o">===</span> <span class="mi">123</span><span class="p">);</span>
    <span class="nx">console</span><span class="p">.</span><span class="nx">assert</span><span class="p">(</span><span class="nx">data</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">myfloat</span> <span class="o">===</span> <span class="mf">123.45</span><span class="p">);</span>
    <span class="nx">console</span><span class="p">.</span><span class="nx">assert</span><span class="p">(</span><span class="nx">data</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="dl">'</span><span class="s1">456</span><span class="dl">'</span><span class="p">]</span> <span class="o">===</span> <span class="dl">"</span><span class="s2">abc</span><span class="dl">"</span><span class="p">);</span>
    <span class="nx">console</span><span class="p">.</span><span class="nx">assert</span><span class="p">(</span><span class="nx">data</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">truebool</span> <span class="o">===</span> <span class="kc">true</span><span class="p">);</span>
    <span class="nx">console</span><span class="p">.</span><span class="nx">assert</span><span class="p">(</span><span class="nx">data</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">falsebool</span> <span class="o">===</span> <span class="kc">false</span><span class="p">);</span>
    <span class="nx">console</span><span class="p">.</span><span class="nx">assert</span><span class="p">(</span><span class="nx">data</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">user_list</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">last_name</span> <span class="o">===</span> <span class="dl">"</span><span class="s2">O'Connor</span><span class="dl">"</span><span class="p">);</span>
    <span class="nx">console</span><span class="p">.</span><span class="nx">assert</span><span class="p">(</span><span class="nx">data</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">user</span><span class="p">.</span><span class="nx">id</span> <span class="o">===</span> <span class="mi">1</span><span class="p">);</span>
    <span class="nx">console</span><span class="p">.</span><span class="nx">assert</span><span class="p">(</span><span class="nx">data</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">date_time</span><span class="p">.</span><span class="nx">toDateString</span><span class="p">()</span> <span class="o">===</span> <span class="dl">"</span><span class="s2">Sat Aug 27 2022</span><span class="dl">"</span><span class="p">);</span>
    <span class="nx">console</span><span class="p">.</span><span class="nx">assert</span><span class="p">(</span><span class="nx">data</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">html</span> <span class="o">===</span> <span class="dl">'</span><span class="s1">&lt;p class="m-100 float-left random modifier p-100 spacer-5 john-b-good"&gt;My &lt;strong&gt;Paragraph&lt;/strong&gt;&lt;/p&gt;</span><span class="dl">'</span><span class="p">);</span>
  <span class="p">}</span>
  <span class="nx">get_data_from_data_attribute</span><span class="p">({{</span> <span class="nx">data</span><span class="o">|</span><span class="nx">to_json</span> <span class="p">}});</span>
<span class="nt">&lt;/script&gt;</span></code></pre></figure>

<p>Keep in mind if you use a Javascript framework like <a href="https://vuejs.org/">Vue</a> or <a href="https://reactjs.org/">React</a>
and you’re wanting to pass your context data into a custom component, then
you’ll need to use <a href="https://docs.djangoproject.com/en/4.1/ref/templates/builtins/#force-escape">force_escape</a>. Example:</p>

<figure class="highlight"><pre><code class="language-html" data-lang="html"><span class="nt">&lt;div</span> <span class="na">id=</span><span class="s">"data-div"</span> <span class="na">data-data=</span><span class="s">"{{{ data|safe|force_escape }}"</span><span class="nt">&gt;&lt;/div&gt;</span>
<span class="nt">&lt;script&gt;</span>
<span class="kd">function</span> <span class="nx">get_data_from_data_attribute</span><span class="p">()</span> <span class="p">{</span>
  <span class="kd">const</span> <span class="nx">divElem</span> <span class="o">=</span> <span class="nb">document</span><span class="p">.</span><span class="nx">querySelector</span><span class="p">(</span><span class="dl">'</span><span class="s1">#data-div</span><span class="dl">'</span><span class="p">);</span>
  <span class="kd">let</span> <span class="nx">data</span> <span class="o">=</span> <span class="nx">divElem</span><span class="p">.</span><span class="nx">dataset</span><span class="p">.</span><span class="nx">data</span><span class="p">;</span>
  <span class="nx">data</span> <span class="o">=</span> <span class="nx">JSON</span><span class="p">.</span><span class="nx">parse</span><span class="p">(</span><span class="nx">data</span><span class="p">)</span>
  <span class="nx">data</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">date_time</span> <span class="o">=</span> <span class="k">new</span> <span class="nb">Date</span><span class="p">(</span><span class="nx">data</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">date_time</span><span class="p">);</span>
  <span class="p">...</span>
<span class="p">}</span>
<span class="nx">get_data_from_data_attribute</span><span class="p">();</span>
<span class="nt">&lt;/script&gt;</span></code></pre></figure>

<h2 id="so-why-not-use-djangos-json_script-template-tag">So why not use Django’s json_script template tag?</h2>

<p>The short answer is you can use the <a href="https://docs.djangoproject.com/en/4.1/ref/templates/builtins/#json-script">json_script</a> template tag, but there are some things to consider.
First, if you look at the <a href="https://github.com/django/django/blob/ae509f8f0804dea0eea89e27329014616c9d4cc0/django/utils/html.py#L62">actual code</a> you’ll see that the filter wraps the code in a script tag
which I’ve found to not be necessary. Secondly, there isn’t a great way to use a subclassed <code class="language-plaintext highlighter-rouge">DjangoJSONEncoder</code> that we
need if there are custom data types that the default <code class="language-plaintext highlighter-rouge">DjangoJSONEncoder</code> doesn’t handle.</p>

<h2 id="what-about-security">What about security?</h2>

<p>You should always be mindful of security and test your application to make sure that it hasn’t created a security
exploit. I also recommend reading Django’s excellent <a href="https://docs.djangoproject.com/en/4.1/topics/security/#cross-site-scripting-xss-protection">security documentation</a>. Having said that, in our
example filter, <code class="language-plaintext highlighter-rouge">to_json()</code> I’m using <a href="https://docs.djangoproject.com/en/4.1/ref/utils/#django.utils.safestring.mark_safe">mark_safe</a>, which I’m fine with because I’ve read and know the
precautions to take. First, under no circumstances pass user-submitted data to the <code class="language-plaintext highlighter-rouge">to_json()</code> filter unless you know
the data has been sanitized with something like <a href="https://github.com/mozilla/bleach">bleach</a>. Secondly, write some tests and test your application
manually for XSS exploits.</p>

<p>I should also mention that in our example data we’re using <code class="language-plaintext highlighter-rouge">User.objects.all()</code> for illustrative purposes only. However,
those with a keen eye might recognize that this would send the hashed password to the template, which I don’t
recommend. Instead, to avoid this I recommend you something like
<code class="language-plaintext highlighter-rouge">User.objects.values_list('first_name', 'last_name', 'email')</code> or something equivalent.</p>

<h2 id="final-thoughts">Final Thoughts</h2>
<p>Hopefully, this post will end up saving engineers a lot of time and prevent a lot of bugs from being created. I also
hope that maybe Django could either include some of this information in the documentation or add a template filter
like <code class="language-plaintext highlighter-rouge">to_json</code> to Django that could be easily extended with your project’s own default encoder.</p>]]></content><author><name>Brent O&apos;Connor</name></author><category term="django" /><summary type="html"><![CDATA[Serializing Django view context data for use in Javascript is something I’ve always thought should be extremely easy, but any time you have data that is more then a simple data type like a string or an integer, it can get complicated pretty quickly and can end up causing hard-to-solve bugs.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://epicserve.com//assets/images/epicserve-default-social-share-logo.png" /><media:content medium="image" url="https://epicserve.com//assets/images/epicserve-default-social-share-logo.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry></feed>