Understanding and Building the Simplest Neural Network

Over the past year I’ve been reading and learning about neural networks, how they work, and how to use them. I’ve found the overwhelming majority of tutorials and introductions to NN either: a) focus on the math and derivations, or b) focus on the code and tools, but rarely seem to c) match the math 1:1 with the code. Both of these tutorial styles are often guilty of handwaving and simplifying the derivations, making it difficult for me to follow exactly how the math and code relate to each other.

An example: neural networks are built in layers of neurons, but why are they built in fully-connected layers?

The primary reason turns out to be performance. It’s dramatically faster to use matrix operations to calculate hundreds or thousands of neurons in parallel on the GPU. Using layers of neurons lends the math to using linear algebra and matrices for a dramatic speedup.

But at the very very very basic mathematical level, a neural network doesn’t have to use layers, it can look like anything:

The neuron-level math works exactly the same, even though the network-level math wouldn’t be able to optimize with matrices. They layered structure is computationally optimized, but not computationally necessary, for building a neural network.

As I was learning, I wanted to separate in my mind what was necessary for a neural network to function vs what was an optimization.

I’ve decided to start writing my own ‘book’ of sorts to document everything I’m learning about neural networking, starting from the very basics. The First Chapter covers the very smallest bits of math and code needed to build the Smallest Possible Neural network. The goal is to start with what is absolutely necessary for neural networks, and nothing more, and slowly build upon that foundation chapter by chapter.

If you’ve been searching for the simplest and most complete primer on neural networking, then download Chapter 1 and signup to be notified as I write the next chapters.

Loose Leaf 3.0.0 is Open Source!

In addition to the great new features in v3.0.0 – the entire app is now open source! Get the code!

Organize Your Notes

This version brings multiple-document organizationt to Loose Leaf. You can setup multiple documents, easily switch between documents, and quickly move and copy pages between documents. Check out the new tutorial videos on getlooseleaf.com.

Import and Export PDFs

Version 2.2.0 brought single page PDF import and export, and now in v3.0.0 you can import entire PDFs into Loose Leaf to read and annotate. With the new multiple-document features above, you can also quickly move pages between PDFs, annotate, and export and share!

Get the Code!

Much of Loose Leaf’s code has already been open sourced before, but today 100% of the codebase is now open source! Check out the project on Github.

Get the App!

Support further development and download the app!

Introducing ClippingBezier: Find find intersecting points, paths, and shapes from UIBezierPath

UIBezierPath’s an an incredibly powerful tool for modeling complex paths and shapes in Objective-C and Swift, but it’s surprisingly difficult to perform operations on two or more paths. Applications like PaintCode make it easy to get the difference or intersection between two paths, but there are limited options for doing this on demand in code. This is particularly meaningful in drawing apps like Loose Leaf, where all of the paths are generated by the user.

Loose Leaf is unique – when the user draws on an imported image, the ink actually sticks to that image. This is done by splitting the UIBezierPath of the user’s pen with the UIBezierPath border of each scrap on the page.

All of this magic is packaged into the new ClippingBezier framework for iOS.

Example: Find intersecting points between two paths

Let’s start with a simple example: finding the exact points that two paths intersect. Below we have two paths, a circle and a square:


The following code give us an array of intersection points, that we can iterate over and mark on our canvas:

NSArray* intersections = [square findIntersectionsWithClosedPath:circle andBeginsInside:nil];
for (DKUIBezierPathIntersectionPoint* intersection in intersections) {
     CGPoint p = intersection.location1;
     [[UIColor redColor] setFill];
     [[UIBezierPath bezierPathWithArcCenter:p radius:7 startAngle:0 endAngle:2*M_PI clockwise:YES] fill];

And now we can mark precisely where these two arbitrary paths intersect.


Example: Find overlap between two paths

Let’s take the above example to the next step: what part of the circle’s path is actually inside of the square, and can we split it apart from the circle itself?

ClippingBezier makes it easy to find these intersections. To make the outputs visual even in code, the paths labeled (in name only) with colors. The blue path defines the clipping path, and the red and green paths are the clipped paths, either inside or outside the blue clipping path respectively


To calculate the above paths, we can use the following code in ClippingBezier:

NSArray* redGreenAndBlueSegments = [UIBezierPath redAndGreenAndBlueSegmentsCreatedFrom:square bySlicingWithPath:circle andNumberOfBlueShellSegments:NULL];
NSMutableArray* redSegments = [redGreenAndBlueSegments objectAtIndex:0];
NSMutableArray* greenSegments = [redGreenAndBlueSegments objectAtIndex:1];
NSMutableArray* blueSegments = [redGreenAndBlueSegments objectAtIndex:2];

void(^drawSegmentsWithColor)(NSArray*,UIColor*) = ^(NSArray<DKUIBezierPathClippedSegment*>* segments,UIColor* color){
	for (DKUIBezierPathClippedSegment* segment in segments) {
		[color setStroke];
		[[segment pathSegment] setLineWidth:2];
		[[segment pathSegment] stroke];

drawSegmentsWithColor(redSegments, [UIColor redColor]);
drawSegmentsWithColor(greenSegments, [UIColor greenColor]);
drawSegmentsWithColor(blueSegments, [UIColor blueColor]);

And that’s it! now we have physically split the circle curve into separate UIBezierPath’s representing the portions inside and outside the square.

Example: Find shapes cut from the intersection of two paths

ClippingBezier can also calculate the sub-shapes generated from the intersecting and overlapping paths. I wrote about the algorithm that ClippingBeizer uses in my post about Loose Leaf’s scissors tool. Now you can build that same feature into your own apps.

With a single line, we can ask ClippingBezier to calculate the sub-shapes generated from the shape and scissors paths.

NSArray* shapes = [square uniqueShapesCreatedFromSlicingWithUnclosedPath:circle];
for (DKUIBezierPathShape* shape in shapes) {
	[[UIColor randomColor] setFill];
	[shape.fullPath fill];
[[UIColor greenColor] setStroke];
[circle stroke];

and now we can iterate over these new sub-shapes and fill them each with a random color.


Example: Cutting Paths with Holes

ClippingBezier also handles UIBezierPaths that contain subpaths – essentially holes in their shapes. The following example shows the same circle path slicing the square with a smaller square hole:

UIBezierPath* square = [UIBezierPath bezierPathWithRect:CGRectMake(200, 200, 200, 200)];
[square appendPath:[[UIBezierPath bezierPathWithRect:CGRectMake(230, 230, 140, 140)] bezierPathByReversingPath]];

With the above square path, the generated sub-shapes now show:


Working with Complex Shapes

ClippingBezier isn’t limited to simple shapes like squares and circles. These simpler shapes make for easy to understand examples, but ClippingBezier was designed to work with any complex Bezier path. Check out the intersections and sub-shapes found by intersecting a complex path with its own mirror


Get the code

ClippingBezier is available on Github under the MIT license.

Thanks for your support!

Support the project and download Loose Leaf from the App Store. Thanks!

Google Author link
Page 1 of 5112345...102030...Last »