Two days ago, I scheduled a call with Brandon,[^brandon] asking to brainstorm together about research questions/directions for my space solar power research.

In the past week or two, I've been skimming through an intro electrical engineering course and reading Prof. Naoki Shinohara's 2014 book *Wireless Power Transfer via Radiowaves*. Though complex circuit diagrams and graphs still went over my head, through the combination of the two I began to get a sense of how the components of microwave power transmission worked: notes on semiconductor HPAs, microwave tubes, phase shifts, rectifier circuits, and what have you filled my notebook.

Yet, after finishing Prof. Shinohara's book and reading his chapter on SSP in another 2018 book, I struggled to find a clear chasm or create connections for the way forwards. Where did the problem, the bottleneck lie? What kind of paper could I write, or research could I do, that would be a meaningful contribution to the pursuit of space solar power?

I found myself bouncing around web articles and videos aimlessly. [In one video](https://www.youtube.com/watch?v=bveGhdtlFn0), I was slightly demoralized to find out that Elon Musk had called SSP the "stupidest idea ever" owing to the amount of conversions taking place in the process compared to terrestrial solar power with battery storage. Eventually, I stumbled upon an [ESA competition for new SSP ideas](https://ideas.esa.int/servlet/hype/IMT?documentTableId=45087625530300097&userAction=Browse&templateName=&documentId=514a8db636ea637f6e27069183966350) that had just closed on January 12, 2021. On the competition page, ESA listed a few challenge areas for SSP (paraphrased):

- Manufacturing, deploying, and constructing very large structures in space
- Onboard energy conversion from solar to electric to RF
- Thermal systems
- Microwave generation, control, focusing, and pointing
- RF to electricity conversion on receiving end

I jotted these down in my notebook without giving too much thought to them.

Twenty minutes before my meeting with Brandon, I pulled them up again.

The latter four bullet points came into my mind, and out again. I had some inklings of the improvements that needed to be made in WPT components -- higher efficiency, lower size and weight, lower cost -- but they didn't seem to be bottlenecks for SSP. Research on long-distance WPT has been done and continues to be done.

The first point, on the other hand, stuck out to me. The big reason we don't have any SSP satellites in orbit today is...we can't launch them. The physics of MPT require a kilometer-scale, or at least several-hundred-meters scale antenna to be deployed in space, for any sort of efficient transfer to the ground from LEO or GEO. Such a satellite can't simply be packed onto a Falcon 9 and launched into space.

Snippets of thought began coming back to me: origami antennas, Caltech's solar panel array deployment research. A quick Google search yielded several articles and papers about the deployment of large-scale structures in space, for SSP and other applications.

Unlike the technical optimizations of WPT, launch and deployment are intuitive problems that need to have rock-solid solutions for space solar power to get anywhere.

"The overall vision has been validated, it's the operational details that need to be worked out," Brandon frames it in our conversation. He brings up what I call the TKS "activator thesis": the vast majority of problems and opportunities in the world don't have theoretical obstacles, but rather execution/activation ones. The discovery of penicillin would have been completely forgotten if not for a US Army physician who discovered it buried in research papers during WWII.

Thus, rather than WPT technology directly, it seems that deployment of large-scale space structures would currently be the most fruitful area of research for realizing the SSP vision. The theoretical research and electric engineering has been demonstrated through a hundred spin-off applications, and will continue to improve. Deployment is the SSP-specific chasm that remains to be crossed. This was actually one of the first thoughts I had after learning about SSP: why hasn't there been a small demonstration satellite thrown up in orbit yet? Solving operational problems to the point where demonstrations can actually be made and momentum created is how innovative visions turn into executable reality.

Perhaps my hypothesis will be proven wrong, and there are significant bottlenecks elsewhere that I'll come to understand. But for now, after a week of somewhat directionless research that has yielded lots of general knowledge, I've stumbled upon a first concrete problem to aim towards.

[^brandon]: TKS New York Director, MIT MechE MS, former SpaceX and Raytheon employee

Activating Space Solar Power

You've probably heard of transistors. They're the fundamental elements that computers are built from. They can be shrunk to a few nanometers; the switch from vacuum tubes to transistors drove the information revolution, and Moore's law famously predicts that the number of transistors in a computer chip doubles every two years.

But what actually are transistors? How do they work? How are complex computers built up from them?

## Diodes

We'll start by understanding how diodes work. You'll see how this relates to transistors in a moment.

Diodes are elements that only allow current to flow one way in a circuit. In the diagram below, the diode in the left circuit is "forward-biased", so current flows and the lightbulb lights up. The diode in the right circuit is "reverse-biased", so no current flows and the lightbulb remains unlit.

![](/blog/2021-01/transistor/diodes.png)

Diodes function using something called the "PN junction." P and N refer to two types of silicon semiconductor material.

On its own, silicon has four valence electrons. In a crystal lattice, silicon forms stable bonds with four other silicon atoms around it. There are no free electrons and thus no conductivity of electricity.

![](/blog/2021-01/transistor/silicon.jpg)

To make silicon useful, we have to inject another kind of atom to allow for free electron flow. This injection is referred to as either "N-doping" or "P-doping".

In N-doping, an atom with five electrons, such as phosphorous, is injected. Four of these electrons form stable bonds with surrounding silicon atoms, but the fifth electron is now free to move around the lattice.

![](/blog/2021-01/transistor/n-doped.jpg)

In P-doping, an atom with three electrons, such as boron, is injected. The three electrons form bonds with surrounding silicon atoms, but now there is a "hole" to which other electrons can move to fill.

![](/blog/2021-01/transistor/p-doped.jpg)

In a diode, N-doped and P-doped silicon are stacked next to each other. The free electrons in the N-doped silicon are attracted to the holes in the P-doped silicon, and start moving across the divide. When they cross the divide, they fill the holes in the P side. Now there's a negative charge on the P side of the divide, and electrons from the N side are repelled from rather than attracted to the P side. Thus, electrons stop flowing to the P side, and at equilibrium a depletion layer is formed with a potential difference of 0.7 V.

![](/blog/2021-01/transistor/diode1.jpg)

When a voltage of greater than 0.7 V is applied across the diode, the potential difference of the depletion layer is overcome, and electrons are able to make the jump from the N to the P side, completing the circuit and creating a current.

![](/blog/2021-01/transistor/diode2.jpg)

When a voltage of the opposite direction is applied, the depletion layer widens: electrons on the N side are attracted to the positive terminal of the voltage source, and holes on the P side to the negative terminal. Thus, there is no electron flow and no current.

![](/blog/2021-01/transistor/diode3.jpg)

Diodes are useful for a variety of purposes, such as rectifying AC to DC power:

![](/blog/2021-01/transistor/rectifier.jpg)

## Bipolar Junction Transistor

We're now ready to talk about transistors! The first of two types of transistors we'll talk about is the bipolar junction transistor, invented by William Shockley at Bell Labs in 1948 (who would win the 1956 Nobel Prize in Physics for 	his research on semiconductors and transistors, along with two other Bell Labs researchers).

A BJT consists of N, P, and another N layer being stacked back-to-back against each other.

When a voltage is applied on the two terminal N layers of the transistor either way, no current flows.

The magic happens when we introduce a second loop of the circuit, connecting one N terminal to the P layer in the middle. Looking at this loop in isolation, we see that it's just the diode circuit we were looking at earlier!

When a forward-biased voltage of greater than 0.7 V is applied in this diode circuit, current flows across the NP junction and completes the top loop of the circuit. Overcoming the depletion layer of the NP junction also closes the bottom loop of the circuit and allows current to flow.

Thus, we see that our transistor is functioning as a switch! When a sufficient voltage, i.e. > 0.7 V, is applied in the top loop of the circuit, the bottom loop is closed. When the voltage is insufficient, the bottom loop is open. Transistors can be much smaller and faster than physical switches, which is what makes them useful for computer chips that cram billions of transistors into a small slab of silicon.

Bipolar junction transistors can also function as amplifiers. An electric signal in the top loop, for example a microphone that introduces voltage fluctuations corresponding to air pressure differences, is reflected in the bottom loop as well. With a larger voltage source in the bottom loop, this signal is thus amplified, and can, for example, now drive a speaker to amplify sound from the microphone.

In a BJT, the three contacts are called the emitter, base, and collector. You can buy a whole tray of transistors for a few dollars, and they're commonly used in hobby electronics projects.

## Field Effect Transistor

For computer chips, though, BJTs are still too bulky. Another kind of transistor, the field effect transistor (FET), are a later development and actually used in integrated circuits and computer chips.

FETs look a little different from BJTs:

![](/blog/2021-01/transistor/fet1.jpg)

Our N and P layers are still present, along with two terminals attached to each of the N layers. Instead of a second loop connected directly to the P layer, though, the third terminal of a FET is the "gate", prevented from contacting the N or P layers directly by an insulator.

![](/blog/2021-01/transistor/fet2.jpg)

As before, a depletion layer is formed between the N and P layers. When a voltage is applied across the two N terminals, no current is able to flow.

When a positive voltage is applied to the gate, the field created eliminates the depletion layer in a region between the two N terminals, allowing electrons to flow and closing the circuit. Thus, as before, the FET functions as a switch.

![](/blog/2021-01/transistor/fet3.jpg)

A particular type of FET, metal-oxide-semiconductor field-effect transistors, or MOSFETs, are credited with revolutionizing the electronics industry and kicking off the age of modern computing. Invented by Mohamed Atalla and Dawon Kahng at Bell Labs in 1959, MOSFETs have since become the most frequently manufactured device in history, with 13 sextillion -- that's 13e21 -- manufactured to date.

To see concretely how transistors are used in computers, consider basic logic gates like AND and OR. From these gates, all components of a computer system -- memory, storage, processing -- can be built. In turn, logic gates can be constructed out of basic transistor circuits:

![](/blog/2021-01/transistor/gates.png)

Thus, with FETs, millions and billions of logic gates can be crammed onto IC chips, memory modules, solid-state storage devices, and more, powering the electronics around us.

How Do Transistors Work? The Building Blocks of Modern Computing

![](/blog/2021-01/updately-notifications/follow.jpg)

My personal favorite feature in [the 0.3.0 release of Updately](https://preview.mailerlite.com/q7e6t2) was comments. Updately is meant to be a social platform, but there was previously no way to interact with others' posts other than viewing them. Discussion was facilitated, but on different platforms. Comments brought Updately one step closer to being a social platform in its own right.

There was a major flaw, though: users had no way of knowing if another user had left a comment on their update, and reminders had to be sent by other platforms anyways. That's because Updately currently doesn't have any sort of notification system.

Today, I'm going to build a notification system for Updately, using a MongoDB collection just like user profiles, updates, and comments. Notifications will be created when comments or replies to comments are left, and users will see their notifications through the frontend when they log in. In the future, notifications can be expanded both ways: they can be created for more use cases, such as mentions in updates or comments, and also delivered to users in more ways, such as through email. For now, though, we'll focus on creating them only for comments, and checking them only through the frontend.

## 1. Setting up the schema

The first step is to create a new collection for notifications. We do this by defining a Typescript type, then a schema that is exported as a Mongoose model.

`types.ts`:

```jsx
import {Document} from "mongoose";

export interface Notification extends Document {
    _id: string,
    userId: string,
    authorId: string,
    updateId: string,
    type: "comment" | "reply",
    read: boolean,
    createdAt: string, // date string
    updatedAt: string, // date string
}
```

`models.ts`:

```jsx
import mongoose, {Schema, ObjectId, Model} from "mongoose";
import {Notification} from "../utils/types";

// helper type object
const reqString = {
    type: String,
    required: true,
};

// define schema
const notificationSchema: Schema = new Schema({
    userId: ObjectId, // ID of receiving user
    authorId: ObjectId, // ID of comment author
    updateId: ObjectId, // ID of update of comment to generate link and notification message
    type: reqString, // "comment" | "reply"
    read: {type: Boolean, required: true},
}, {
    timestamps: true,
});

// export as model
export const notificationModel Model<Notification> = mongoose.models.notification || mongoose.model('notification', notificationSchema);
```

Originally this schema actually looked quite different: I had `message` and `link` string fields instead of `updateId`, `authorId`, and `type`. This means that the notification can be displayed as soon as the object is loaded. However, this runs into problems if the user or update information changes, which would change both the message and link for the notification. Attaching IDs and types to the notification and letting the frontend handle message and link generation is therefore much more robust.

## 2. Create notifications on comment creation
Comments are created by hitting `/api/new-comment` with a request containing this body info:

```jsx
{
	commentText: commentText,
	authorId: userData._id,
	updateId: update._id,
	updateAuthorId: update.userId,
	commentId: parentCommentId,
}
```

The API endpoint then uses Mongoose to create the comment. Right afterwards, we can use Mongoose to create the appropriate notifications:

```jsx
mongoose.connect(process.env.MONGODB_URL, {
  useNewUrlParser: true,
  useUnifiedTopology: true,
  useFindAndModify: false,
});

const thisUser = await userModel.findOne({ email: session.user.email });

const newComment = {
  body: req.body.commentText,
  authorId: new mongoose.Types.ObjectId(thisUser._id),
  updateId: new mongoose.Types.ObjectId(updateId),
  isSubComment: !!commentId,
  parentCommentId: commentId ? new mongoose.Types.ObjectId(commentId) : null,
};

const returnComment = await commentModel.create(newComment);

// if comment author is not update author, create notification for update author
if (req.body.updateAuthorId !== thisUser._id.toString()) {
  const newNotification = {
    userId: req.body.updateAuthorId,
    updateId: updateId,
    authorId: thisUser._id,
    type: "comment",
    read: false,
  };
  await notificationModel.create(newNotification);
}

// if comment is subcomment, create notifications for authors of all subcomments of parent comment
if (newComment.isSubComment) {
  const parentComment = await commentModel.findOne({ _id: commentId });
  const subComments = await commentModel.find({ parentCommentId: commentId });
  const commentUserIds = [
    parentComment.authorId.toString(),
    ...subComments.map((d) => d.authorId.toString()),
  ]
    .filter((d, i, a) => a.indexOf(d) === i) // filter out duplicates
    .filter(
      (d) => d !== thisUser._id.toString() && d !== req.body.updateAuthorId
    ); // filter out ID of comment and post author
  for (let userId of commentUserIds) {
    const newNotification = {
      userId: userId,
      updateId: updateId,
      authorId: thisUser._id,
      type: "reply",
      read: false,
    };
    await notificationModel.create(newNotification);
  }
}

```

Now, if I go to my sister's update and post a comment, a notification document is added to the notifications collection in MongoDB Atlas.

![](/blog/2021-01/updately-notifications/posting-comment.jpg)

![](/blog/2021-01/updately-notifications/posting-comment-atlas.jpg)

## 3. Fetch notifications

First, we'll create a new API route `get-notifications` that fetches notifications for the user currently logged in. With Next.js, this is as easy as adding a file `get-notifications.ts` to `/pages/api`:

```jsx
export default async function getNotificationsHandler(req: NextApiRequest, res: NextApiResponse) {
    if (req.method !== "GET") return res.status(405);
    const session = await getSession({ req });
    if (!session) return res.status(403).json({message: "You must be signed in to fetch notifications"});

    try {
        const thisUser: User = await getCurrUserRequest(session.user.email);
        const notifications: Notification[] = await getNotifications(thisUser._id.toString());
        if (notifications.length === 0) return res.status(200).json({notifications: [], users: [], updates: [], updateUsers: []});
        else {
            const uniqueAuthorIds: string[] = notifications.map(d => d.authorId).filter((d, i, a) => a.indexOf(d) === i);
            const uniqueUpdateIds: string[] = notifications.map(d => d.updateId).filter((d, i, a) => a.indexOf(d) === i);
            const users: User[] = await userModel.find({ "_id": {$in: uniqueAuthorIds}});
            const updates: Update[] = await updateModel.find({ "_id": {$in: uniqueUpdateIds}});
            const uniqueUpdateUserIds: string[] = updates.map(d => d.userId).filter((d, i, a) => a.indexOf(d) === i);
            const updateUsers: User[] = await userModel.find({ "_id": {$in: uniqueUpdateUserIds}});
            return res.status(200).json({notifications: notifications, users: users, updates: updates, updateUsers: updateUsers});
        }
    } catch (e) {
        res.status(500).json({error: e});
    }
}

export async function getNotifications(userId: string) {
    await mongoose.connect(process.env.MONGODB_URL, {
        useNewUrlParser: true,
        useUnifiedTopology: true,
        useFindAndModify: false,
    });

    return notificationModel.find({ userId: userId });
}
```

The above function follows a similar pattern as comments. Comment documents themselves only contain update and user IDs, while the user information itself is needed to render the comment in the frontend. Instead of having this fetching happen on the frontend, the API call itself returns an array of all associated user objects, which can then be matched up in the frontend to render the comment. The same is done for fetching relevant update and user objects for notifications here.

It's easy now to use `useSWR` in our navbar component to fetch notifications:

```jsx
const { data: notificationsData, error: notificationsError } = useSWR(session ? "/api/get-notifications" : null, fetcher) || {data: null, error: null};
```

## 4. Display notifications

First, I added a simple notification bell:

```jsx
{
  notificationsData && (
    <button className="mr-4 p-2 relative">
      <FiBell />
      {notificationsData.notifications.length > 0 && (
        <div className="rounded-full w-3 h-3 bg-red-500 top-0 right-0 absolute text-white font-bold">
          <span style={% raw %}{{ fontSize: 8, top: -9 }}{% endraw %} className="relative">
            {notificationsData.notifications.length}
          </span>
        </div>
      )}
    </button>
  );
}

```

Then, I added a dropdown menu to it, using some utility classes I had already defined:

```jsx
{
  notificationsData && (
    <button className="mr-4 px-2 h-10 relative up-hover-button">
      <FiBell />
      {notificationsData.notifications.length > 0 && (
        <>
          <div className="rounded-full w-3 h-3 bg-red-500 top-0 right-0 absolute text-white font-bold">
            <span style={% raw %}{{ fontSize: 8, top: -9 }}{% endraw %} className="relative">
              {notificationsData.notifications.length}
            </span>
          </div>
          <div className="up-hover-dropdown mt-10 w-64">
            {notificationsData.notifications
              .sort((a, b) => +new Date(b.createdAt) - +new Date(a.createdAt))
              .map((notification: Notification) => {
                const thisUpdate: Update = notificationsData.updates.find(
                  (d) => d._id === notification.updateId
                );
                const thisUpdateUser: User = notificationsData.updateUsers.find(
                  (d) => d._id === thisUpdate.userId
                );
                const thisAuthor: User = notificationsData.users.find(
                  (d) => d._id === notification.authorId
                );

                return (
                  <MenuLink
                    text={
                      {
                        comment: (
                          <>
                            <span>
                              <b>{thisAuthor.name}</b> commented on your{" "}
                              {format(new Date(thisUpdate.date), "M/d/yy")}{" "}
                              update
                            </span>
                            <br />
                            <span className="opacity-50">
                              {formatDistanceToNow(
                                new Date(notification.createdAt)
                              )}{" "}
                              ago
                            </span>
                          </>
                        ),
                        reply: (
                          <>
                            <span>
                              <b>{thisAuthor.name}</b> replied to your comment
                              on
                              {" " +
                                (thisUpdateUser.email === session.user.email
                                  ? "your"
                                  : thisUpdateUser._id === thisAuthor._id
                                  ? "their"
                                  : thisUpdateUser.name + "'s") +
                                " "}
                              {format(new Date(thisUpdate.date), "M/d/yy")}{" "}
                              update
                            </span>
                            <br />
                            <span className="opacity-50">
                              {formatDistanceToNow(
                                new Date(notification.createdAt)
                              )}{" "}
                              ago
                            </span>
                          </>
                        ),
                      }[notification.type]
                    }
                    href={`/@${thisUpdateUser.urlName}/${thisUpdate.url}`}
                    nowrap={false}
                  />
                );
              })}
          </div>
        </>
      )}
    </button>
  );
}

```

```
.up-hover-dropdown {
    display: none;
    @apply absolute top-0 right-0 z-40 shadow-lg rounded-md bg-white;
}

.up-hover-button:hover > .up-hover-dropdown, .up-hover-button:focus > .up-hover-dropdown {
    display: block;
}
```

And voila! We have a notification dropdown that works great on desktop and mobile. Clicking the notification takes you to the update with the comment.

![](/blog/2021-01/updately-notifications/bell2.jpg)

![](/blog/2021-01/updately-notifications/notif.jpg)

![](/blog/2021-01/updately-notifications/notif2.jpg)

## 5. Mark notifications as read
First, I made an api route `/api/read-notifications`, to which a notification ID can be posted to mark it as read.

```jsx
import {NextApiRequest, NextApiResponse} from "next";
import {notificationModel} from "../../models/models";
import mongoose from "mongoose";
import {getSession} from "next-auth/client";
import {getCurrUserRequest} from "../../utils/requests";
import {Notification, User} from "../../utils/types";

export default async function readNotificationsHandler(req: NextApiRequest, res: NextApiResponse) {
    if (req.method !== "POST") return res.status(405);
    const session = await getSession({ req });
    if (!session) return res.status(403).json({message: "You must be signed in to mark a notification as read"});
    if (!req.body.id) return res.status(422).json({message: "No notification ID found in request"});

    try {
        console.log(req.body.id);

        const thisUser: User = await getCurrUserRequest(session.user.email);
        const thisNotification: Notification = await getNotification(req.body.id);

        if (!thisNotification) return res.status(404).json({message: "No notification found for given ID"});
        if (thisNotification.userId.toString() !== thisUser._id.toString()) return res.status(403).json({message: "You do not have permission to mark this notification as read"});

        thisNotification["read"] = true;
        await thisNotification.save();
        res.status(200).json({message: "Successfully marked notification as read"});
    } catch (e) {
        res.status(500).json({error: e});
    }
}

export async function getNotification(notificationId: string): Promise<Notification> {
    await mongoose.connect(process.env.MONGODB_URL, {
        useNewUrlParser: true,
        useUnifiedTopology: true,
        useFindAndModify: false,
    });

    return notificationModel.findOne({ _id: notificationId });
}
```

At first, I considered adding an `onClick` handler to the notification link, which Next.js allows to execute before redirecting. This would be easier than parsing anything additional on the navigated to page, I thought. Using an `onClick` handler would cause a wait time before navigation, though. And I soon realized a pretty easy way to have the notification be marked as read on the navigated to page: pass the notification ID as a query param, and run a `useEffect` function to mark the notification as read if it's present.

The link in `navbar.tsx` now looks like this: 

```jsx
href={`/@${thisUpdateUser.urlName}/${thisUpdate.url}?notification=${notification._id}`}
```

And in `[updateUrl].tsx`, a useEffect hook that looks like this:

```jsx
useEffect(() => {
  if (router.query.notification) {
    axios
      .post("/api/read-notification", {
        id: router.query.notification,
      })
      .then((res) => {
        console.log(res);
      })
      .catch((e) => {
        console.log(e);
      });
  }
}, [router.query.notification]);
```

Now we just need to adjust the `navbar` frontend code to display read notifications differently. First I calculate a new variable, the number of unread notifications: `const numNotifications = notificationsData ? notificationsData.notifications.filter(d => !d.read).length : 0` (`notificationsData` is undefined as `useSWR` fetches it).

Now, I only display the badge on the bell if `numNotifications` is above zero:

```jsx
{
  numNotifications > 0 && (
    <div className="rounded-full w-3 h-3 bg-red-500 top-0 right-0 absolute text-white font-bold">
      <span style={% raw %}{{ fontSize: 8, top: -9 }}{% endraw %} className="relative">
        {numNotifications}
      </span>
    </div>
  );
}
```

I also wrapped each notification menu item in a div making the notification half-opacity if it's read:

```jsx
<div key={notification._id} className={notification.read ? "opacity-50" : ""}>
  <MenuLink ... />
</div>
```

With that, we have, more or less, a complete working notification system!

![](/blog/2021-01/updately-notifications/half-opacity.jpg)

## 6. Clear out old notifications

As users get notifications, it'll build up endlessly both on the user side menu and in the database, which we don't want. As a simple fix, whenever a user fetches their notifications, we'll delete all read notifications older than 14 days. This is straightforward to implement with a few array operations:

```jsx
...

// delete read notifications older than 14 days
const oldNotificationIds: string[] = notifications
  .filter((d) => d.read && +new Date() - +new Date(d.createdAt) > 14 * 8.64e7)
  .map((d) => d._id.toString());
await notificationModel.deleteMany({ _id: { $in: oldNotificationIds } });

notifications = notifications.filter(
  (d) => !oldNotificationIds.includes(d._id.toString())
);


...
```

## 7. Conclusion & extensions
That's it for this basic notification system!

There are plenty of improvements to be made. `useSWR` doesn't re-fetch notifications after one is clicked, for example, which is a simple matter of passing a key variable to the hook to fix. If there are a lot of notifications, there's no height limit or overflow on the notification menu, but again this is a simple fix using CSS.

It's easy to extend this system to new types of notifications, too. For example, I added notifications for when users follow each other.

Hopefully this post is a helpful example of how you might implement your own notification system using MongoDB!

![](/blog/2021-01/updately-notifications/follow.jpg)

Implementing Notifications Using MongoDB and Next.js for Updately

![](/blog/2021-01/edyfi-elo/final.jpg)

At the community house [Edyfi](https://edyfi.co/85e37d59e0064c8d8f4f2cb91168eb18), we have a ping pong table. As soon as quanratine ended, we started playing a lot of ping pong.

Ben and Luke put together a roll of white board paper on the wall to track their ping pong games. Over the course of the day, the rest of us added our games as well.

It got chaotic pretty quickly, and I got to thinking -- what if we had an Elo rating system for the house? Matches would have actual stakes attached. This is day two of ping pong being available and we'll be here for two more months, figuring out a rating system felt like a good investment.

Immediately, my mind went to creating a React app. Ben recommended we look for an existing tool first. We found [https://github.com/altuzar/pingpong-elo-ranking](https://github.com/altuzar/pingpong-elo-ranking), but it was a little jank (requiring each player to make an account and log in to log games). At the end of the day, I was surprised to come up short, without a good existing solution. So, building our own it was.

I got as far as typing `create-next-app` into my terminal before restraining myself again. React is so heavy, surely there was a way to do this without React. We would need a database, too. Here, my mind went to AirTable. I set up a table and put in existing games pretty quickly, with a couple of nice fields like auto-incrementing IDs, creation datetime, and winning player determined by score.

![](/blog/2021-01/edyfi-elo/airtable.jpg)

I then set about looking how to calculate Elo through AirTable, but confronted with the difficulty of using formulas across tables and the heft of developing an AirTable app to use scripts, I gave up and decided to use AirTable's API and handle calculations separately.

Considering that AirTable's API requires an authentication token to access, my browser-JS dream went out the window too. I needed to set up serverside functions of some sort, as well as a frontend. Put those needs together, and the best tool by far is...NextJS. I hate to use something as heavy as React for something this simple, but devX trumps stack efficiency for a quick build like this. Let's get into it then!

## Accessing AirTable
First, I created a `.env` file and threw my AirTable key in it. Nice and secure, and NextJS makes this zero-config.

In typical create-next-app fashion, I deleted the `/api` folder, `/public` items, `Home.module.css`, and the contents of `index.js`. I'll leave `globals.css` to have some easy styling.

`npm i airtable` gets me AirTable's JS library. Now it's easy to fetch info for all games using `getServerSideProps`:

```jsx
import Head from 'next/head';
import airtable from "airtable";

export default function Home({games}) {
    console.log(games);

    return (
        <>
            <Head>
                <title>Edyfi ping pong Elo tracker</title>
            </Head>
        </>
    );
}

export async function getServerSideProps(ctx) {
    const base = airtable.base("appJW8IinnjFjPllg");
    const games = await base("Games").select().all();
    const gamesFields = games.map(d => d.fields);
    return {props: {games: JSON.parse(JSON.stringify(gamesFields))}};
}
```

![](/blog/2021-01/edyfi-elo/browser.jpg)

## Calculating Elo
With some nice ES6 array operations, we first get a list of unique players from the game records, then initialize an object with each player's Elo rating:

```jsx
// all unique player names
const allPlayers = [...games.map(d => d["Player 1"]), ...games.map(d => d["Player 2"])]
		.filter((d, i, a) => a.indexOf(d) === i);

// initialize dictionary with each player at rating 1000
let playerElos = Object.fromEntries(allPlayers.map(d => [d, 1000]));

console.log(playerElos);
```

![](/blog/2021-01/edyfi-elo/ratings.jpg)

Now we just need to run through all the games and use the Elo formula to calculate the Elo for each player:

![](/blog/2021-01/edyfi-elo/elo-formula.png)

(Credit: [https://www.coorpacademy.com/en/blog/learning-innovation-en/elo-whos-the-best/](https://www.coorpacademy.com/en/blog/learning-innovation-en/elo-whos-the-best/))

```jsx
// maximum amount rating can be changed per game
const maxChange = 30;

for (let game of games.sort((a, b) => +new Date(a["Date"]) < +new Date(b["Date"]))) {
        const rating1 = playerElos[game["Player 1"]];
        const rating2 = playerElos[game["Player 2"]];
        const expected1 = 1 / (1 + 10 ** ((rating2 - rating1) / 400))
        const expected2 = 1 / (1 + 10 ** ((rating1 - rating2) / 400))
        const score1 = +(game["Win"] === game["Player 1"]);
        const score2 = +(game["Win"] === game["Player 2"]);
        const newRating1 = rating1 + maxChange * (score1 - expected1);
        const newRating2 = rating2 + maxChange * (score2 - expected2);
        playerElos[game["Player 1"]] = newRating1;
        playerElos[game["Player 2"]] = newRating2;
}
```

![](/blog/2021-01/edyfi-elo/elos.jpg)

## Displaying Elo and games
First, let's refactor the earlier calculations into a state variable and `useEffect` hook:

```jsx
const [playerElos, setPlayerElos] = useState(function(){
        // all unique player names
        const allPlayers = [...games.map(d => d["Player 1"]), ...games.map(d => d["Player 2"])]
                .filter((d, i, a) => a.indexOf(d) === i);

        // initialize dictionary with each player at rating 1000
        return Object.fromEntries(allPlayers.map(d => [d, 1000]));
}());

useEffect(() => {
        // maximum amount rating can be changed per game
        const maxChange = 30;

        let newPlayerElos = {...playerElos};

        for (let game of games.sort((a, b) => +new Date(a["Date"]) < +new Date(b["Date"]))) {
                const rating1 = newPlayerElos[game["Player 1"]];
                const rating2 = newPlayerElos[game["Player 2"]];
                const expected1 = 1 / (1 + 10 ** ((rating2 - rating1) / 400))
                const expected2 = 1 / (1 + 10 ** ((rating1 - rating2) / 400))
                const score1 = +(game["Win"] === game["Player 1"]);
                const score2 = +(game["Win"] === game["Player 2"]);
                const newRating1 = rating1 + maxChange * (score1 - expected1);
                const newRating2 = rating2 + maxChange * (score2 - expected2);
                newPlayerElos[game["Player 1"]] = newRating1;
                newPlayerElos[game["Player 2"]] = newRating2;
        }

        setPlayerElos(newPlayerElos);
}, []);
```

For styling, I went with [`water.css`](https://watercss.kognise.dev/), a classless CSS framework. From here, it's super easy to bash out a few HTML tables and get the rest of the site going:

```jsx
<Head>
    <title>Edyfi ping pong Elo tracker</title>
    <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/water.css@2/out/water.css"/>
</Head>
<h2>Player stats</h2>
<table>
    <thead>
        <tr>
            <th>Rank</th>
            <th>Name</th>
            <th>Elo</th>
        </tr>
    </thead>
    <tbody>
        {Object.entries(playerElos)
            .sort((a, b) => b[1] - a[1])
            .map((player, i) => (
                <tr>
                    <td>{i + 1}</td>
                    <td>{player[0]}</td>
                    <td>{Math.floor(player[1])}</td>
                </tr>
            ))
        }
    </tbody>
</table>
<hr/>
<h2>Post new game</h2>
<p><a href="https://airtable.com/shrl6FUmJspGANrPK">+ Post new game through AirTable form</a></p>
<hr/>
<h2>Past games</h2>
<table>
    <thead>
        <tr>
            <th>Date</th>
            <th>Player 1</th>
            <th>Score 1</th>
            <th>Player 2</th>
            <th>Score 2</th>
        </tr>
    </thead>
    <tbody>
    {games.sort((a, b) => +new Date(b["Date"]) - +new Date(a["Date"])).map(game => (
        <tr>
            <td>{format(new Date(game.Date), "M/d 'at' h:mm aa")}</td>
            <td>{game["Player 1"]}</td>
            <td>{game["Player 1 score"]}</td>
            <td>{game["Player 2"]}</td>
            <td>{game["Player 2 score"]}</td>
        </tr>
    ))}
    </tbody>
</table>
```

I'll stop here for this basic implementation, but there's much more that can be done with this interface (some updates, like win/loss counts, are shown in the final image at the top of this post)! Per-game Elo, specific matchup records, player win/loss records, etc. That's just a matter of more lines of code, though, so I won't document it that thoroughly.

![](/blog/2021-01/edyfi-elo/prototype.jpg)

From here, it was a super easy one-click deploy to Vercel, and the site was live for all the members of the house to use. Productivity has since tanked.

Nothing super fancy going on here, but it's pretty amazing how easy it is to bash out a full webapp in less than an hour with today's web technology and services.

Creating a Ping Pong Elo Tracker for Edyfi in 1 Hour

TKS Community Talks Round 1 Post-Mortem

I built and launched a social platform app in three days. Here's how I did it, and what I learned

Understanding the math behind neural networks by building one from scratch (no TF/Keras, just numpy)

![](/blog/2020-09/gd-cover.png)

If you've taken a lab science class in school, you've probably had to fit a line of best fit to experimental data: whether it's to experimentally determine the acceleration of gravity, calculate the results of a chemical reaction, or prove that two variables are correlated. You plug your numbers into a spreadsheet, hit "fit trendline," and out pops a nice linear or exponential equation. But how does Excel or Google Sheets come up with this equation?

This problem is that of training a linear regression model. There are lots of statistics-related theorems and considerations behind it, but this article will focus on an algorithm that computers use to actually find the best fit equation given a dataset. The same principles that empower Excel to find a line of best fit are the fundamentals for a variety of machine learning algorithms and applications, from **deep neural networks** to **recommendation systems.** **Understanding linear regression is a great first step to understanding all the other cool ML algorithms out there — and it's not even that hard!** Let's dive in.

*Note: all graphs, calculations, and even data used in this article were created by me in Python. Check out the source code in [this Jupyter notebook](https://colab.research.google.com/drive/1fwNB1WH03KnEq221Xe1grJeYmkx46pi6?usp=sharing).*

## The basics: mean squared error cost function

Here's a dataset to fit a line to.

![](/blog/2020-09/Untitled.png)

If you've taken Algebra, you should be familiar with the equation for a line:

$$y = mx + b$$

By tweaking m and b, we can conjure up any line that we want to. We can take a few guesses at a line of best fit for our dataset:

![](/blog/2020-09/Untitled%201.png)

Clearly, some of these lines fit the data better than others. Yellow looks the best — green is too steep, and blue isn't steep enough. But how can we know for sure that yellow is better than green and blue? And how can we find not just a good line, but the best line possible?

We can start by coming up with a number to measure how well a line fits the given data. We can use a metric called the mean squared error.

To calculate the mean squared error, take every example and measure the vertical distance between it and the line above or below. Square this distance, then average the squares of all examples.

![](/blog/2020-09/Untitled%202.png)

Here's the formula for calculating mean squared error for our linear equation:

{% raw %}
$$J = \frac{1}{2 \cdot \text{# examples}} \sum_{i = 1}^\text{# examples} ((mx^{(i)}+b) - y^{(i)})^2$$
{% endraw %}

where $$x^{(i)}$$ and $$y^{(i)}$$ are the i-th pair of x and y values in our dataset.

The lower the MSE is, the better the line fits the data. This should make sense — all that MSE is measuring is the average squared distance from the data points to the corresponding predicted value on the line. The smaller this distance, the better the line.

We can use a simple loop in Python to calculate the mean squared error of the three lines we guessed above:

```python
def mean_squared_error(m, b, x_data, y_data):
	num_examples = x_data.size
	J = 0

	for i in range(num_examples):
		J += ((m * x_data[i] + b) - y_data[i]) ** 2

	J = J / (2 * num_examples)

	return J

print(mean_squared_error(1, 50, x_data, y_data))
print(mean_squared_error(2, 50, x_data, y_data))
print(mean_squared_error(4, -50, x_data, y_data))
```

We get that the MSE for the first line — our blue, too low line — is 2070.76. The MSE for the second line — our yellow, seems-best line — is a mere 375.26. The MSE for the third line — the green, too-steep line — is back up to 2280.60.

This confirms our visual findings from before. Our yellow line, with the equation $$y = 2x + 50$$, has the lowest MSE and fits the data much better than the two other lines. By this metric, our blue line is also worse than our green line.

With this metric, we're one step closer to a systematic way to find a line of best fit. We can now guess a bunch of lines, and then check their mean squared error, keeping the one with the lowest MSE and maybe tweaking it some more.

This is random and inefficient, though. Luckily, there's a much better, systematic way of finding the best possible line once we've defined our cost function (MSE).

## Finding the optimal solution: gradient descent

Let's return to one of our bad lines:

![](/blog/2020-09/Untitled%203.png)

If we want to improve it, what should we do? A lot of the points look like they're above the line, so let's increase b to 70.

![](/blog/2020-09/Untitled%204.png)

Looks a little better, and our MSE has dropped by almost 40%, from 2071 to 1286! Let's keep going and increase b to 90.

![](/blog/2020-09/Untitled%205.png)

Another drop, to MSE = 902. Now let's try b = 110:

![](/blog/2020-09/Untitled%206.png)

...and now we've overshot it, with our MSE increasing to 918.

To find the optimal value of b and not overshoot, we might graph the MSE against different values of b.

![](/blog/2020-09/Untitled%207.png)

Looks like the optimal value is just below 100. If you've taken algebra or calculus, your instinct at this point might be to bust out some derivatives or quadratic equations and try to find out exactly what this point is — but it's not so easy, because we don't actually have the equation for this line. Every one of those points above was computed by looping through 100 training examples and averaging their errors.

Instead of finding the optimal value analytically, then, we do it numerically.[^normaleq] These are fancy names for saying that instead of trying to solve an equation exactly, we'll use clever tricks to arrive at the right answer by changing our answers just a little bit, over and over.

Let's go back to our starting point at b = 50. Let's plot the slope of our b vs. MSE plot at b = 50:

![](/blog/2020-09/Untitled%208.png)

We can find the slope at a given point using this formula:

$$\frac{\delta J}{\delta b} = \frac{1}{\text{# examples}} \sum_{i = 1}^\text{# examples} ((mx^{(i)}+b) - y^{(i)})$$

If you've taken calculus, you can take the derivative of our earlier mean squared error function (J) with respect to b yourself and see if your answer matches. If you don't know calculus, just accept that this equation accurately gives us the slope at a given point. Look into derivatives separately if you want.

The point is, once we have this slope, we can nudge b an amount proportional to the steepness of the slope. Think of it like rolling down a hill. If the hill is steeper, you'll roll down more. If it's shallower, you'll roll down less. If you keep rolling down a hill like this, you'll eventually get to the lowest point of the valley — and in this case, our ideal value for b.

In equation form, this is what we're doing:

$$\text{repeat } i \text{ times } \{

b := b - \alpha \frac{\delta J}{\delta b}

\}$$

It might look a little confusing, but all we're doing here is what we said earlier: nudging b an amount proportional to the steepness, or gradient, of the graph of the cost function at that point. $$\alpha$$, or alpha, is called the "learning rate" — it's a multiplier on how much you slide down the hill, something like the strength of gravity or the slipperiness of the hill. If alpha is big, you'll slide down the hill quickly. If it's small, you'll slide down it bit by bit. As you slide towards the bottom of the valley, the gradient will flatten out and steps will become smaller. The value that you're optimizing will then converge to the value that results in the lowest error — the best fit b for your data, in this case.

With a learning rate of 0.5, we roughly converge to a b value around 99 in less than ten steps:

![](/blog/2020-09/Untitled%209.png)

We've only found the optimal value for b, however. Our MSE is still above 800, more than double that of our initial best-guess line, 2x + 50. We could now set b to be 99 and optimize for m, but that wouldn't get us the best answer either. To get the actual line of best fit, we have to optimize for both m and b at the same time.

To do this, we can plot J against both m and b using a contour map:

![](/blog/2020-09/Untitled%2010.png)

![](/blog/2020-09/Untitled%2011.png)

You may have seen contour maps of a mountain or particular region before. Such a map shows the land elevation at a given point. The contour map above does a very similar thing: it shows the mean squared error for every pair of $$m$$ and $$b$$ values. By finding the $$m$$ and $$b$$ values that give us the lowest possible mean squared error, then, we'll have found our line of best fit.

The intuition here — and the math — is the same as before: we find the slope (gradient) at a given point, and roll down the hill (change our parameters) an amount proportional to how steep the slope is. When we calculate the gradient at a certain point, we no longer have just one value, but two values, corresponding to how the cost function changes when you nudge either m or b.

The formula for calculating these two gradient values at a given point are as follows:

$$\frac{\delta J}{\delta b} = \frac{1}{\text{# examples}}\sum_{i = 1}^\text{# examples} ((mx^{(i)}+b) - y^{(i)}) \\
\frac{\delta J}{\delta m} = \frac{1}{\text{# examples}}\sum_{i = 1}^\text{# examples} ((mx^{(i)}+b) - y^{(i)})x$$

Before, we subtracted the gradient with respect to b, times a learning rate, from b to get our new value. We'll do that again here, and we'll do the same separately for m, *at the same time*:

$$\text{repeat } i \text{ times }\\\{\\
b := b - \alpha \frac{\delta J}{\delta b} \\
m := m - \alpha \frac{\delta J}{\delta m}
\\\}$$

Just as before, we can illustrate this two-variable gradient descent visually on our contour map and 3D plot. I plotted gradient descent from three starting points (red: m = 3, b = 150; green: m = 1, b = -200; blue: m = 1, b = 103).

![](/blog/2020-09/Untitled%2012.png)

As you can see, it doesn't matter where you start — gradient descent will eventually lead you to the right answer. In fact, standard practice when training machine learning models is to use small random values for your starting points.

In this case, following the gradient, we find an approximate global minimum at m = 2 and b = 49. Translating back from the world of equations and graphs, this means that the line (approximately) $$y = 2x + 49$$ results in the lowest mean squared error when measured against our dataset, and consequently is the **line of best fit for the dataset**[^theorem], what we've been looking for all along.

Phew! After all that hard work, we're finally done. Let's do a quick recap of what just went down.

- We defined a cost function, something called the "mean squared error" that measures how far the predictions of our model are from the actual data.
- Our objective is to minimize the cost function (in fact, another name for the cost function is the "objective function").
- We started this by graphing the cost function against one parameter, in this case the y-intercept (b) of the line.
- To minimize the cost function, we "nudged" the parameter in the direction that would decrease the cost.
- We used gradient descent is a way to make these nudges systematically, by choosing them proportional to the slope, or gradient, of the graph. It's like rolling downhill!
- We then included our second parameter, in this case the slope of the line, in our graph of the cost function. It can now be visualized as a contour plot, or a 3D surface.
- We can apply the same method as earlier — gradient descent, sliding downhill with the slope of the 3D surface — to optimize both parameters at once. This gives us the optimal parameters m and b, yielding our line of best fit.

This might seem like a lot to take in, especially the mathematical equations. I know it took me a good few learning days to gain a sense of comfort with them. Understanding the math behind how to fit a trendline, though, is incredibly useful for understanding tons of other machine learning algorithms out there.

## Beyond linear regression: a fundamental concept in machine learning

The general pattern of defining a cost function and using gradient descent to optimize for it is ubiquitous in supervised learning algorithms. Fitting a polynomial function to a dataset rather than a straight line, for example, is as easy as defining the cost function as the following:

$$J = \frac{1}{2 \cdot \text{# examples}} \sum_{i = 1}^\text{# examples} ((\theta_0 + \theta_1 x^{(i)} + \theta_2 x^{(i)2}) - y^{(i)})^2$$

where we now have three parameters theta 0, theta 1, and theta 2 to optimize rather than just m and b. In fact, we can throw as many more parameters as we want at the cost function, and very often optimize hundreds or thousands of parameters in just this way.

Instead of a line of best fit (a regression model with continuous outputs), we can train a classifier using the very same cost function-gradient descent strategy. We do this by running the continuous model output through a logistic (or sigmoid) function, which outputs a number between 0 and 1 that we can interpret as a probability of the input belonging to a certain class.

Neural networks use cost functions and gradient descent too, though the cost function and parameter optimization methods are a bit more complicated, involving in-between feed-forward and backpropagation algorithms.

Even some unsupervised learning problems — that is, models where there is no "right" answer that can be provided in training data — use cost functions and gradient descent. When training a recommendation algorithm, like Netflix or Amazon might use to recommend you movies and products, a cost function is defined that simultaneously optimizes for parameters fitting users' preferences and the categorization of different items.

It's easy to just say these things. You might even have heard of gradient descent, and the analogy of sliding down a hill, before reading this article. For me at least, though, it's much more helpful to see the concrete math. I like learning how an algorithm works well enough to be able to code it from scratch, without using TensorFlow or scikit-learn or whatever is out there. Of course, if I'm working on an actual project, I'll use the better-optimized and more advanced tools that these libraries have to offer; even then, though, I'm guided by the intuition I gained from going very hands-on into the math.

A lot of the math isn't super hard, either. In this article, for example, the most advanced things that you need to understand are summation loops and...squares. Sure, there were a few derivatives, but I didn't tell you what they meant, because you don't need to know to build your core mathematical machine learning intuition. This relative mathematical simplicity extends, again, beyond linear regression to most of the common machine learning algorithms in use out there. Neural networks are really just *a lot* of summation loops strung together cleverly. If you fully understand the contents of this article, you're already well on your way to being able to write a neural network from scratch yourself.

To get a rigorous but accessible overview of the math behind gradient descent, linear regression, neural networks, recommendation systems, and everything that I mentioned in this article, I highly recommend Andrew Ng's [free Machine Learning course](https://www.coursera.org/learn/machine-learning) on Coursera. People's minds work differently — maybe trying to understand the math won't be especially helpful for you, but for me it built the foundation of all of my applicable machine learning intuition and knowledge today. Maybe this article will convince you to start building this foundation for yourself :)

[^normaleq]: There actually is a way to find the optimal value analytically, using something called the normal equation. This equation puts all data points in a big matrix, then uses some fancy linear algebra to exactly find the linear regression parameters that minimize the mean squared error. Because it's multiplying matrices together, though, if you have lots of data or lots of variables, the normal equation can become very slow to compute compared to gradient descent.)

[^theorem]: We know that minimizing mean squared error results in the parameters for the best fit line because of the [Gauss-Markov theorem](https://en.wikipedia.org/wiki/Gauss%E2%80%93Markov_theorem?oldformat=true).

How Excel calculates trendlines, and how you can too: an introduction to gradient descent, the most important Machine Learning algorithm

<a href="https://www.producthunt.com/posts/sz-project-tracker" target="_blank" style="border-bottom: 0;">
	![](/blog/2020-07/szpt-ph.jpg)
	</a>

*Also posted on [IndieHackers](https://www.indiehackers.com/post/from-fearing-react-to-ph-webapp-launch-in-2-months-8b9f102b7c) and [DEV.to](https://dev.to/wwsalmon/from-fearing-react-to-ph-webapp-launch-in-2-months-4d58). See the project case study [here]({{ site.baseurl }}/sz-project-tracker)*

Four months ago, I didn't know React or any other JS framework. In fact, having done tons of static web dev work, I was terrified of React. How do you make an entire website in JavaScript??

Still, I wanted to go beyond making landing pages and WordPress templates. I made several attempts at learning React and some backend stuff like the MERN stack. In theory, I know how to spin up a MERN app with a CRUD API and all, but none of these tutorials really stuck with me. I was left not knowing how to deploy the local apps and servers I had built, not knowing how to extend it into something usable.

Eventually, I found my way to serverless-stack.com. This was a huge tutorial that teaches you how to set up an AWS backend (Cognito, DynamoDB, Lambda proxy API), as well as a React frontend connected through it. It didn't teach how to use these tools so much as just use them, throwing new functions and AWS services at you every other page. This is how I learn best, though -- from blitzing through full React page after React page, I started to pick up a better intuition than I had from any tutorial before. I finished the tutorial series in about a week and a half in May.

Throughout the process, I tracked my learning and progress in a markdown file committed to the repo. I did the same for another project. It was really useful to have screenshots and code snippets from previous versions to reference in blog posts, or just for personal reflection. Using a nice, lightweight markdown editor was the best solution I found for jotting down these quick updates, but I kept thinking about how nice it would be to have an app properly designed for this. I looked online, but surprisingly couldn't find one. Well, good thing I've been learning how to make one!

So that's the project that I jumped into: an app to track your projects, making dev and learning logs 10x easier to keep than with a traditional or markdown document editor. I actually ditched the Serverless framework as I was taught in serverless-stack, instead using AWS Amplify CLI to super quickly get all my services up and running; still, serverless-stack served as my main guide, for backend as well as frontend reference.

At some point in the project, I outgrew serverless-stack for both frontend and backend. Frontend came first. Serverless-stack used class components, while newer tutorials used functional components; my code was a hodgepodge of both at the beginning. [Josh Kaufman says that, contrary to the 10,000 hours of learning => expert rule, you can learn something *reasonably* well in 20 hours](https://www.youtube.com/watch?v=5MgBikgcWnY). I remember almost tangibly *feeling* this kind of 20 hour mark. I split off my code into components and lib functions, refactored class components into functional ones with confidence. A partner joined the team, and I was surprised to realize how much better I could recognize good or bad React code in PRs just weeks after not knowing React at all.

Backend came much later -- really in the last week, even. I was scared of the Amplify CLI. There were so many config files, with so many parameters and variables. Every push I made came with the warning that things might break, that data might be lost. So I worked with the backend as little as I could at first. I set up auth, storage, API, and just left it; API schema updates were all I dared to run. I couldn't run away forever, though. Two features, both procrastinated until this very last week, forced me to actually learn. One was making images attached to public updates publicly viewable, as they were uploaded into private S3 buckets. My nicely protective Amplify functions weren't enough for this. Now I had to deal with ACLs, using the full AWS SDK, dealing with credentials and auth tokens directly -- diving straight into the backend config stuff that I feared most. The second feature was adding users to a Mailchimp mailing list on signup. Mailchimp has a nice API, but it can't be called on the clientside, so I had to set up a Lambda function for it. I decided to do this on pre-launch day. "This should be quick," I thought at 2PM; at 11:20PM, having coded discreetly through a meeting, it finally seemed to work. I asked a friend to sign up with her Google account, and felt a huge sense of relief when her email popped up in the Mailchimp audience list.

Progress on this project was quite sporadic, as side-projects often are. I would blitz forwards for a week, knocking out feature after feature and squashing bug after bug, then the rest of my life would catch up to me and I would take a lull. When a partner and mentor joined the team, progress actually slowed down; caught in thinking about containerized environments, CI/CD, delegating work, etc., I lost the motivation that came from tangible progress. I was also consistently working in unfamiliar territory. I had no idea how to solve or even approach many of the problems I came up against; I would stare for hours at documentation and tutorial videos, writing test functions, my frustration growing as I didn't seem to make an inch of progress. Out of many, many repeated such experiences, though, I found that this frustrated deadlock often *did* eventually result in learning and progress. "Micro-framework: let yourself be stuck on things for three hours. Stare at tutorials, write test functions, don't make progress; struggle & learn and eventually it just might click," [I tweeted after one session](https://twitter.com/wwsalmon/status/1280620786664038405). One potential implication, then, is that this kind of learning and from-scratch skill building can't be done in small chunks, but rather require continuous long blocks of time. (Not entirely sure if this is true, maybe it would actually be sped up if broken apart? To be tested, but the value of long continuous blocks of work time is a popular idea)

One day, I was tired of this project sitting around, draining me of my energy as the rest of my life and what I could be doing flew by.

"We're gonna launch in two weeks, by Wednesday, July 8," I told my partner. Planning out features, we frequently said things like, "this isn't necessary for v1, let's implement it for v2." It's the classic "cut scope rather than extending the deadline" -- and it worked. I had a target. My energy was back. I refactored code, redesigned UIs, and added small missing features like crazy, in the course of four or five days sprinting across the remaining distance to a mostly functional, fairly reliable app.

An online community/program I've been a part of, [Summer of Shipping](https://summerofshipping.com/), has demo opportunities at meetings every Thursday. My goal was to demo that Thursday; I grabbed a few screenshots, wrote some copy, and put together a quick landing page (so easy with @rmrm's a17t and Tailwind CSS). My demo was full of bugs (Google sign-in redirected to localhost instead of live site, Twitter URL opened as relative instead of absolute...everything goes wrong in the demo), but it was mostly done and out there!

The idea of a ProductHunt launch was really only in the back of my mind until pretty late on. I had previously launched a [Chrome extension, YouTube Liberation](https://www.producthunt.com/posts/youtube-liberation) on PH. My friends and everyone I reached out to raved about how useful the extension was, but after hyping myself up, greating a fancy animated thumbnail, choosing the day to post to get maximum views, I got all of...three upvotes. For this project, my expectations were basically non-existant. I built it for myself; I hadn't showed many other people, nor did anyone seem especially like they were jumping for it. Still, I had pretty much already made everything I needed when making the landing page. It'd be minimal effort to just hop in Illustrator, turn these into graphics, and toss it up on PH, I thought. This time I at least have the Summer of Shipping network -- maybe it'll get a few signups.

Yesterday morning, I fixed the bugs from the demo, filled out the PH fields, and was about to hit launch when I remembered that PH allowed you to launch through a hunter. It just so happened that @swyx, senior dev advocate at AWS working on Amplify and a huge advocate for the "build in public" mentality that Summer of Shipping and my project tracker alike build upon, had given a talk at Thursday's SoS meeting and seen my demo. I messaged him on Twitter, sending over my prepared slides, asking him if he would be willing to hunt the product for me. I wasn't sure if he would -- were my product, my hastily assembled marketing materials good enough? Would he harshly dismiss me? No! He said yes!

I sent him some more materials. He sent me...bugs that he encountered when he signed up for the app. "Today will be a bit of a bug squashing marathon 😅," I messaged him. "Better prelaunch than postlaunch," he replied.

I had a bunch of other work I was planning to yesterday. For a design job, I needed to put together mockups by Saturday; for an entrepreneurship program, conduct a bunch of customer interviews. Instead of doing this, I ended up spending the whole day fixing bugs and making minor improvements. I changed the url from the abysmal sz-project-tracker-v0.netlify.app to the slightly less abysmal szpt.netlify.app. I wanted a mailing list of people who signed up, so I made a Mailchimp account and looked into their API. This, oh god, this was 8 hours straight of the standstill blundering-my-way-through learning that I had talked about earlier.

Everything was bashed out. The PH page was ready. At midnight, it went live. I woke up at 8:30 AM this morning. The PH page had 8 upvotes. Already broke my 3 upvote record, lmao...but I thought I could do better. I posted on Twitter; I asked @swyx, [@irid](https://twitter.com/coolnalu), and some others to repost; I posted this very blog post, which was originally intended to be mostly promotional but turned into the much more substantial story it is now, on [IndieHackers](https://www.indiehackers.com/edit-post/8b9f102b7c) and [DEV.to](https://dev.to/wwsalmon/from-fearing-react-to-ph-webapp-launch-in-2-months-4d58).

As of writing this post, SZPT has 47 upvotes on ProductHunt, dropping from 13th, to 14th, to 15th on the homepage as the day went on, with 27 new signups on the app. It's not huge -- this app isn't "taking off," so fortunately my AWS bill won't either -- but it's *a good amount* of people using an app that's my first-ever webapp, that I buitl for myself. My only hope is that they'll find legitimate value in it, and value in me as a creator of products.

This is the culmination of two full months of learning and building, from being terrified of JavaScript and writing about the death of hand-written CSS to falling in love with React and CSS frameworks like Tailwind and a17t. Other than technical skills, what have I learned? I think two big ideas have been heavily affirmed:

- "Do to learn, not to do." This is a long-held philosophy of mine, and it's really been the driving force for getting to where I am today. I didn't go to any bootcamps or follow any curriculums in the past two months, but I've made myself so much more valuable by just diving in and trying to make things that I care about (my GitHub got me hired as frontend developer at an early-stage startup!).

- Prioritize connections, people, and communities. Did going to Summer of Shipping talks about the inner workings of authentication or HTTP/CORS help me build my app? No, but it gave me a group of people to be a part of; my work provided value for the community, so mentors could invest their time in me. Indirectly, SoS had a huge impact on my network too, by pushing me to get on Twitter, and creating opportunities for engagement through mentors' and the community's accounts. Through SoS and more broadly on Twitter, I've connected with so many amazing people, building a network of experts and industry professionals as well as people working on their own projects and careers like me, who have kept me going and offered incredible insights and advice when I reached out to them.

These, really, are my two superweapons. They've empowered me so much; the realm of possibilities in front of me seems unbounded. What's next for me? We'll see how szpt goes, maybe I'll work on it a bit more. In the longer term,  I have an amazing mentor pushing me to find something that I would want to work on for the next 5-7 years of my life; I'm chasing lots of other opportunities to keep learning, doing, connecting.

I hope you enjoyed reading this post. In my IndieHackers and DEV.to posts, I spammed this pitch for the app all throughout the article. In this more retrospective, chill blog post, I'll just leave it here at the end:

Whether it's to build in public, keep track of work for reports, or just to learn and reflect better, we could all benefit from logging our projects. SZ Project Tracker makes keeping dev logs or learning journals as easy as posting to Twitter 💨

📝 Write in markdown, drop images in
🌎 Make projects & updates public in just a click

▶▶▶ https://producthunt.com/posts/sz-project-tracker

From fearing React to Product Hunt webapp launch in 2 months!

![](/blog/2020-04/raise3.png)

What is recursion? Simply a function that calls itself. As an example, here’s a recursive algorithm for calculating factorials:

```js
function factorial(n) {
    if (n == 2) {
        return 2;
    }
    return n * factorial(n - 1);
}
```

This is a pretty trivial application; the same alg is easy enough to implement with a simple for loop. But for certain complex operations like merge sort, a recursive approach can be much easier and faster.

To be honest, I never thought I’d use recursion outside of my intro compsci class, at least not directly for my own algorithms. I’ve mainly done front-end development — the complexity of my JS functions rarely went beyond a for loop; if we’re fancy, then maybe some multidimensional arrays and manipulation of them. Recursion didn’t seem like something that was useful for functions that weren’t ridiculously complex or specific.

Recently I’ve been working a lot with d3.js, building a charting library for *The Phillipian*’s annual [State of the Academy project](http://pdf.phillipian.net/2019/05102019.pdf). It’s been an amazing project, learning and getting comfortable with d3.js and Javascript in general, as I’ve only used it tangentially in service of non-JS static content before; this project gives me a good starting point for getting more into JS with tools like React. But I digress. d3.js is relevant because using it requires so much more direct and creative programming. It gives you tons of power by aiding with manipulating data and corresponding elements, but you’re leaving behind the world of document flow and generally built-in layout functionality. Coordinates and dimensions are set directly, so you need to come up with your own algorithms for dealing with overlap and even alignment.

As such, the comfort I gained working with arrays on seemingly arbitrary problems has actually come in handy. One such thing, unexpectedly, was recursion! Let me explain.

Here’s the starting point. Nice simple stackedBarChart, and I was just starting to implement labels:


![](/blog/2020-04/startingpoint.png)


The code so far is straightforward. First, I create a polyline, for now with just two points, going from the center of each bar to a certain amount above it. Then, above each of these lines, I render the appropriate text label, and push the right bound of each text label to an array. These right bounds are used to calculate the third point of the polyline, ending right where the text does. (Not sure why it’s not working for “Both” in the example above whoops)

```js
chartGroups.selectAll(".sota-stackedBarChart-label-aboveBar-line")
    .data(d => d)
    .join("polyline")
    .attr("class", "sota-stackedBarChart-label-aboveBar-line")
    .attr("points", d => {
        let x1 = margin.left + x(d[1]) + x(d[0]) / 2;
        let y1 = barHeight / 2;
        let x2 = x1;
        let y2 = -labelBelow;
        return `${x1},${y1} ${x2},${y2}`;
    })
    .attr("stroke-width", separatorStrokeWidth)
    .attr("stroke", lineColor)
    .attr("fill", "none")

let labelRightBounds = [];

chartGroups.selectAll(".sota-stackedBarChart-label-aboveBar-text")
    .data(d => d)
    .join("text")
    .attr("class", "sota-stackedBarChart-label-aboveBar-text")
    .text((d, i) => `${valueLabels[i]}: ${d3.format(".1f")(d[0])}%`)
    .attr("x", d => margin.left + x(d[1]) + x(d[0]) / 2)
    .attr("y", function(d) {
        labelRightBounds.push(d[1] + this.getBBox().x + this.getBBox().width)
        return -2 * labelBelow;
    })
    .attr("alignment-baseline", "bottom")

chartGroups.selectAll(".sota-stackedBarChart-label-aboveBar-line")
    .data(d => d)
    .join("polyline")
    .attr("points", function(d, i) {
        let currentPoints = this.getAttribute("points");
        return currentPoints + ` ${labelRightBounds[i]},${-labelBelow}`;
    })
```

So, okay! Clearly the labels overlapping is a problem. A reference from previous print versions of the survey provide guidance on how to fix this:


![](/blog/2020-04/ref1.png)


Not the above/below thing — we’ll keep all the labels above for simplicity. But the “Yes, on campus” and “Yes, off campus” labels provide a perfect model for how to offset the labels so they don’t overlap: raise the height of the first overlapping label, and move the second overlapping label right until it no longer overlaps. Time to code it!


![](/blog/2020-04/raise1.png)


Whoops, that’s not right!


![](/blog/2020-04/raise2.png)


There we go.

This implementation came with a few notable changes from the previous code. I realized that only the text needed to be rendered to calculate the third point of the polyline, and to discover any overlaps, so instead of first creating a polyline with two points and adding the third later, I started by adding the text in their appropriate default spots.

This time, more than the right boundary of each text label is pushed to the `labelRightBounds` array, but rather an array of both its left boundary position and its width. This allows us to calculate overlap between labels, which I do in a straightforward for loop. Iterating from first to last, if the current element overlapped with the previous one, then I would raise the height of the previous one and extend the left position of the current element.

```js
let labelRightBounds = [];

chartGroups.selectAll(".sota-stackedBarChart-label-aboveBar-text")
    .data(d => d)
    .join("text")
    .attr("class", "sota-stackedBarChart-label-aboveBar-text")
    .text((d, i) => `${valueLabels[i]}: ${d3.format(".1f")(d[0])}%`)
    .attr("x", d => margin.left + x(d[1]) + x(d[0]) / 2)
    .attr("y", function(d) {
        labelRightBounds.push([this.getBBox().x, this.getBBox().width]);
        return -2 * labelBelow;
    })
    .attr("alignment-baseline", "bottom")

let labelHeights = []
let prevRightBound = -labelLeft;

for (let i in labelRightBounds) {
    if (labelRightBounds[i][0] < prevRightBound + labelLeft) {
        labelRightBounds[i][0] = prevRightBound + labelLeft;
        labelHeights[i - 1] -= 1;
        labelHeights.push(-2);
    } else {
        labelHeights.push(-2)
    }
    prevRightBound = labelRightBounds[i][0] + labelRightBounds[i][1]
}

chartGroups.selectAll(".sota-stackedBarChart-label-aboveBar-line")
    .data(d => d)
    .join("polyline")
    .attr("class", "sota-stackedBarChart-label-aboveBar-line")
    .attr("points", (d, i) => {
        let x1 = margin.left + x(d[1]) + x(d[0]) / 2;
        let y1 = barHeight / 2;
        let x2 = x1;
        let y2 = (labelHeights[i] + 1) * labelBelow;
        let x3 = labelRightBounds[i][0] + labelRightBounds[i][1];
        let y3 = y2;
        return `${x1},${y1} ${x2},${y2} ${x3},${y3}`;
    })
    .attr("stroke-width", separatorStrokeWidth)
    .attr("stroke", lineColor)
    .attr("fill", "none")

chartGroups.selectAll(".sota-stackedBarChart-label-aboveBar-text")
    .data(d => d)
    .join("text")
    .attr("x", (d, i) => labelRightBounds[i][0])
    .attr("y", (d, i) => labelHeights[i] * labelBelow);
}
```

Iterating from front to back is nice because previous right boundary shifts are accounted for; i.e. the shifting of a previous label might cause it to overlap with the next one, which will now have to move more. But thinking about this alg reveals that it doesn’t actually account for not just one, but multiple labels in a row overlapping, because if a label is found to be overlapping with a previous label, only the height of this immediately preceding label is raised. If this preceding label was overlapping with its preceding label, this precending label would not get raised another step, as it should if the second label were raised by the third.

To make the heights work, you’d have to iterate from last to first label, raising each one a successive step higher if necessary; but then you wouldn’t be able to account for rightward shifts causing overlap, which propagate from first to last label. To solve this problem we’d have to iterate both forwards and backwards.

One solution is to do just that — loop over everything twice: once first-to-last to apply rightward shifts, and again last-to-first to apply height shifts. But before I could get to that, a different idea took hold. **Recursion!**

Let’s take a look at the final implementation — just the recursive part replacing the for loop, as the rendering code didn’t really change.

```js
function getLabelHeight(i) {
    if (i == labelRightBounds.length - 1) {
        labelHeights[i] = -2;
        return -2;
    } else if (labelRightBounds[i][0] + labelRightBounds[i][1] + labelLeft > labelRightBounds[i + 1][0]) {
        labelRightBounds[i + 1][0] = labelRightBounds[i][0] + labelRightBounds[i][1] + labelLeft;
        let nextHeight = getLabelHeight(i + 1);
        let thisHeight = nextHeight - 1;
        labelHeights[i] = thisHeight;
        return thisHeight;
    } else {
        getLabelHeight(i + 1);
        labelHeights[i] = -2;
        return -2;
    }
}

getLabelHeight(0);
```

We still have our `labelHeights` and `labelRightBounds` arrays, the latter storing the left coordinate and width of each label. We define our recursive function function `getLabelHeight` with parameter `i`, the index of the current label in consideration.

Our first conditional checks for the base case: if the label in consideration is the last label. In this case, we set the corresponding `labelHeights` slot to the default height, and return it (or at least the coefficient of it; -2 in this case because the line is -1 units up, and the text -2. This coefficient increments by one from here on out).

If this isn’t the case, then the next label exists, and the current label either overlaps with it or not. If it doesn’t overlap, then easy, we set the corresponding `labelHeights` slot to the default value and return it. Before we do that, we call `labelHeights(i+1)` so the next labelHeight is calculated.

If it does overlap, or is closer than a specified margin, then we must raise the height of the current label. What do we raise the current label to? To one step higher than the height of the next label. But we don’t know the height of the next label, do we? Let’s find it, by calling the very function `getLabelHeight` on it, passing in `i+1` as the parameter; then we can set the return value of the function as the corresponding `labelHeights`  slot, and return this value for the preceding function call. In the case that the next label is raised because it overlaps with the label after that, it will return a raised height, and the height of the current label will be raised to the appropriate height above that. This applies for however many overlapping labels you have; they might clip out of the SVG Canvas, but the heights will cascade all the way from either the first non-overlapping label or the last one back, forming a nice staircase of heights.

Sounds good. Does it actually work?


![](/blog/2020-04/raise2.png)


Okay, nothing changed, but that was what was supposed to happen. That means that the function works for a two-overlap scenario. Let’s shrink down our window to see the cascade in action:


![](/blog/2020-04/raise3.png)


Hell yeah, there we go!

The recursive function ended up being 16 lines of code, compared to the single loop’s 9; implementing the second backwards loop for the same functionality as the recursive function probably would have at least doubled the line count, so the recursive function is certainly nicer to write, and arguably understand, than a double loop. Is it faster to run? I have no idea. In effect it’s looping over the data the same way: once forward, each function call calling the function on the next label before it returns, up until the last label; and then once backward, each function return propagating to the previous function call for the last operations before turning to the next previous call, all the way up to the first call which returns the value to nothing.

None of this was particularly mind-blowing, but it brought me a lot of joy to use what I had learned forever ago in a classroom on a real project, a neat trick that solved a real, somewhat tricky problem.

Offsetting Overlapping Labels in d3.js with a Recursive Algorithm, or: using recursion on a real project for the first time eeeee

Tag: #project