Skip to content

Commit

Permalink
captions
Browse files Browse the repository at this point in the history
  • Loading branch information
JacobReynolds committed Feb 21, 2024
1 parent f4cf865 commit bb52567
Show file tree
Hide file tree
Showing 11 changed files with 20 additions and 28 deletions.
8 changes: 4 additions & 4 deletions beep-boop/foundation/derivatives/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ Which if we take the derivative of $$f(x)$$ where $$x = 3$$ we get $$f'(x) = 6x

This is important, because as we start making complex chains of nodes (a neural net), we want to know the derivative of the output wrt the individual nodes. That way we can tell how we have to change that node to manipulate the output of the function.

![Screenshot showing gradient nodes](./derivatives_nodes.png)
![](./derivatives_nodes.png "Screenshot showing gradient nodes")

This is a good example, where we would want to know the derivative of L wrt c, so we know how to change c to impact L.

Expand Down Expand Up @@ -89,7 +89,7 @@ The chain rule, in simple terms, says that if you have a composite function $$f(

More simply, as we navigate through back propagation, we will want to identify how one node affects the loss function. But this node could be one of many.

![Screenshot showing nodes](./chain_rule_grad.png)
![](./chain_rule_grad.png "Screenshot showing nodes")

Using the above example, the local derivative of $$c \space w.r.t \space d$$ is 1, because it’s [addition](#addition). But how does $$c$$ affect $$L$$? The [Chain Rule](https://en.wikipedia.org/wiki/Chain_rule) gives us this. And what it says is, using the above variable names

Expand All @@ -98,15 +98,15 @@ $$
$$

Which in this case is $$-2 \cdot 1 = -2$$. Why is $$\frac{dd}{dc}=1$$? See [addition](#addition). It's even nicer because as we get further along the back propagation we only need to multiply the local derivative by the gradient of the grandparent node. Take the following:
![Larger screenshot showing nodes](./chain_rule_ext.png)
![](./chain_rule_ext.png "Larger screenshot showing nodes")
If we want to do $$\frac{dL}{da}$$ we would only need to do $$\frac{de}{da} \cdot \frac{dd}{de}$$, since it's all multiplication we don't have to chain it all the way back to L.

Another cool thing here is that since the back propagation only relies on multiplying the local derivative with the grandparent derivative, the calculation can be arbitrarily complex. It doesn't have to be $$+$$ or $$*$$ between nodes, it can be anything as long as we can derive it. This is helpful when doing back propagation on a [neuron](../neuron) that has an activation function, so we can derive the value inclusive of the activation function.

### Addition

Doing the derivative of an additive operation w.r.t. a node is pretty easy. It equals 1. Why is that true? Take these nodes for example
![Screenshot showing nodes](./chain_rule_grad.png)
![](./chain_rule_grad.png "Screenshot showing nodes")

$$
d = c + e
Expand Down
2 changes: 1 addition & 1 deletion beep-boop/foundation/hyper-parameters/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,4 +39,4 @@ plt.plot(lri, lossi)

```

![Line graph showing learning rate decay](./learning-rate-decay.png)
![](./learning-rate-decay.png "Line graph showing learning rate decay")
2 changes: 1 addition & 1 deletion beep-boop/foundation/loss/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ You calculate this by subtracting the actual output of the neural network from t

The curious thing to me is that we don't actually take the mean of the summated squared losses, at least not in anything I've seen so far. So I'm hoping to figure that out. It seems like the division by $$N$$ doesn't really matter, it's the squaring of the loss values that actually give us our metrics. Everything else is just syntactic sugar. This is a popular method when performing regression.

![Mathematical expression of mean squared loss](./mean-squared-loss.png)
![](./mean-squared-loss.png "Mathematical expression of mean squared loss")

## Example

Expand Down
4 changes: 2 additions & 2 deletions beep-boop/foundation/model-types/character-level/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ for _ in range(10):
# teda.
```

![Bigram screenshot](./bigram.png)
![](./bigram.png "Bigram screenshot")

We generate statistics for each character pairing, then sample from those statistics to create terrible names.

Expand Down Expand Up @@ -146,7 +146,7 @@ It took me awhile to get my head around exactly what was happening when you do `

The hardest thing for me to grasp in this is the embedding space. In the following code, you'll see we have a hyper parameter called `EMBEDDING_SPACE` which is 2. This means that for every character `a-z.` in our input, that character lives in 2d space and can be plotted with an x,y axis. When visualizing this, the groupings of these characters represent how closely the neural network thinks they're related.

![2d visualization of the embeddings](./embeddings.png)
![](./embeddings.png "2d visualization of the embeddings")

When we increase the embedding space, we're essentially giving the neural network more dimensions to tune to gauge similarity between the input characters. So it can relate them amongst a variety of commonalities.

Expand Down
2 changes: 1 addition & 1 deletion beep-boop/foundation/multi-layer-perceptron/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ An MLP consists of many layers of [neurons](../neuron/) lined up in order and fe

Since I'm very code inclined, here's the python that implements the following image:

![a multilayer perceptron](./mlp.jpeg)
![](./mlp.jpeg "a multilayer perceptron")

The following code also uses the `Value` class from [Back Propagation](../back-propagation/)

Expand Down
4 changes: 2 additions & 2 deletions beep-boop/foundation/neuron/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ grand_parent: beep boop
# Neuron

Neurons are exactly what they sound like, the things in our brain!
![Diagram of a neuron](./neuron_model.jpeg)
![](./neuron_model.jpeg "Diagram of a neuron")

In machine learning, we model these in neural networks to simulate how the brain works. Neurons take in a series of values (x) and weights (w), which are individually multiplied and then added. The neuron fires by taking these and adding the bias of the neuron (how trigger happy it is) and passing it through an [activation function](#activation-functions) which helps squash the values to something like -1 to 1. This is usually `tanh` or a `sigmoid` function.

Expand All @@ -31,7 +31,7 @@ Much like [weights](#weights), biases are also randomly chosen and updated throu

## Activation functions

![Image showing common activation functions](./activation_functions.png)
![](./activation_functions.png "Image showing common activation functions")
Activation functions are called on the dot product of the weights and their inputs plus the bias of a neuron. This function helps us control the types of outputs we want to come from our neuron.

When using activation functions its important to look at how the weights of that neuron affect the activation function. For example if you're using $$tanh$$ which squashes your values to $$[-1,1]$$ and you have a lot of values $$>10$$ or $$<10$$ you'll have a lot of values that are $$-1$$ and $$1$$ which basically makes the neuron useless. You can [visualize this in PyTorch](../../frameworks/pytorch#visualize-activations) quite nicely as well.
Expand Down
4 changes: 2 additions & 2 deletions beep-boop/frameworks/pytorch/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ If you read the word pytorch over and over, it really starts to lose it's meanin

Here's a simple tree showing the equation and back propagation we're doing, alongside the torch code to calculate the same.

![DAG of nodes for a math equation](./equation.png)
![](./equation.png "DAG of nodes for a math equation")

```python
import torch
Expand Down Expand Up @@ -254,4 +254,4 @@ plt.figure(figsize=(10,20))
plt.imshow(a.abs()>.99, cmap='gray', interpolation='nearest')
```

![Visualizing an activation function](./activation-function.png)
![](./activation-function.png "Visualizing an activation function")
2 changes: 1 addition & 1 deletion business-model/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ icon: briefcase

# 💼 Business models

![Business handshake](./business.gif)
![](./business.gif "Business handshake")

Ahh yes, business. The reason we all get to wear silly little clothes and get up early every day. A big part of starting my own company (TBD) was being able to create an environment that matched the way I believe a business should be run. One that fits the way I want to live my life in general. This is a surprisingly huge problem area that I don't think enough founders spend time to think about before building their start up. Everyone wants to shoot for the moon, make a billion dollars, but forget they have to live a life during all of that as well.

Expand Down
13 changes: 4 additions & 9 deletions cloud-security/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,16 +36,13 @@ So that's it then, be wiz.io for the little guys. Offer enterprise security tool

One of the things that I have going for me is I use to be a hacker (I'd argue I still am, but would never win a CTF at this point). I want to find users of Render and Supabase and ask them about their cybersecurity needs. In cybersecurity, we call this Open Source Intelligence (OSINT) and luckily for me, man is there a lot of it for this use case. For someone to use Render or Supabase they have to update their DNS records and point their domains to ones owned by the cloud provider. I could go around scraping and abusing DNS all around the world, or I could just use our friends over at [host.io](https://host.io). You can look up a domain name or [Autonomous System Number](https://www.cloudflare.com/learning/network-layer/what-is-an-autonomous-system/) (ASN) and see all domain names that point to it.

<video width="100%" alt="Video showing a user accessing host.io" controls>
<source src="./RenderNameLookup.mp4" type="video/mp4">
Your browser does not support videos.
</video>
[!embed el="embed"](./RenderNameLookup.mp4)

After buying $500 worth of API tokens, I was able to start pulling all of these domain names down and prioritize who to reach out to. There were 27k domains, so how can I identify legitimate companies and not small blogging websites? In an ideal world, I'd use [zoominfo](https://zoominfo.com) but they don't really service companies of my size. Luckily for me, [Hubspot](https://hubspot.com) exists. I was able to sign up for their Starter Suite for only $240/year and get access to a pretty sweet set of tools. From there I imported all the domain names and hubspot automatically correlated the domain names to businesses, providing me their sector, revenue, employee count, and more. Unfortunately a lot of this data was pretty hit and miss, it would end up correlating small businesses on the platform to major providers. I doubt your local community center is doing $1,000,000,000 in revenue. I spent a good few days parsing through this data in a variety of ways.

After finding an eligible business, I would look it up on [LinkedIn Sales Navigator](https://business.linkedin.com/sales-solutions/sales-navigator) and reach out to the most senior technical leadership or founder with a message along the lines of:

![Linkedin message asking for advisors](./LinkedinOutreach.png)
![](./LinkedinOutreach.png "Linkedin message asking for advisors")

Sales is a funky thing for me and I imagine most technical people. It can come off as pretty ingenuine and scammy. In most cases, it usually is, so that's a fair thought. Like most things though, I think if you're a good person, working with good people, everyone benefits. To me it felt weird sending hundreds of LI connections asking for advisors, but I routinely found that when I got on a call with someone they were kind and gracious and happy to help. I'm immensely grateful to the people that have taken me up on these offers in their various forms.

Expand All @@ -59,10 +56,8 @@ Side-note: this is a common thing you'll see in my writing and exploration. I've

I realized I shot pretty far downstream from AWS to Supabase, so I look at who might be in the middle. It turns out there's a pretty mature middle market for cloud providers, comprised mainly of companies like [OVHCloud](https://us.ovhcloud.com/), [Vultr](https://www.vultr.com/), and [Hetzner](https://www.hetzner.com/). Wash, rinse, and repeat. I went to host.io (finally putting that $500 to work) and started pulling customers for these providers. Immediately, I'm motivated by seeing how much larger their client base is from the previous providers.

<figure class="small">
<img src="./OVHCloud.png" alt="OVH Cloud has 7 million+ domains pointing to them"/>
<figcaption>7 million OVHCloud domains</figcaption>
</figure>
![OVH Cloud has 7 million+ domains pointing to them](./OVHCloud.png "7 million OVHCloud domains")

And that was another week, scraping, triaging the information, reaching out on LinkedIn, and talking with customers of those platforms. Again, I was surprised that even at this level, aside from some whales, there didn't seem to be a large client base that would be spending enterprise dollars on security. I found a lot of wordpress hosting providers, game server hosting companies, the odd tech company, but no one I met with said that security was in their budget. I'm sure the people are out there, but I couldn't find them. Even now I'm doubting myself and thinking I should go back and keep trying to find those companies, but moving fast is important and I felt this was a big enough barrier to warrant moving on.

## Thanks for the fish
Expand Down
5 changes: 1 addition & 4 deletions index.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,6 @@ Even more concerning to some people, I did so with no idea about what I wanted t

I kind of hate the current hustle culture of audience building, twitter slams, and whatnot, but I do still love writing. I've always enjoyed being able to express myself in a format outside of my head, so this isn't my attempt at building a substack that pulls in $5/mo, but to document my attempts at building something I love, and sharing some of the cool learnings along the way. Grammar be damned, I'll at least get some thoughts out there.

<figure>
<img src="./office.jpeg" />
<figcaption>My small home away from home</figcaption>
</figure>
![My small home away from home](./office.jpeg "My small home away from home")

To begin, I've leased a 60 ft<sup>2</sup> (5 1/2 m<sup>2</sup>) windowless co-working space to trap myself in 8 hours a day and try some cool shit. See ya soon.
2 changes: 1 addition & 1 deletion vulnerability-management/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Looking across the marketplace I don't see anything for vulnerability management

## Ruh roh

![Scooby doo meme](./scooby-doo.jpg)
![](./scooby-doo.jpg "Scooby doo meme")

So here we go again, Sales Navigator and cold outreach. I sent out probably 200 LinkedIn connection invites and talked to ~10 Vulnerability Management leads at large companies that I knew would have budget for security tooling. Every. fucking. single. one. of. them. uses Service Now (SNOW). Not a single one uses Jira for vuln management. There's a wall bridging security and development, and it appears security is behind the iron curtain of SNOW. I've worked with SNOW in the past and have no interest in building an entire company around it. Luckily, again, instead of building an app on Forge for 3 months and not selling it at all, I felt confident enough to disqualify this idea. Like everything, I know there are people out there that use Jira for vulnerability management and pen testing and would benefit from this, but you've got to read the tea leaves for what they are at that point.

Expand Down

0 comments on commit bb52567

Please sign in to comment.