Miniflow

Miniflow is a library to build a small neural networks using NumPy for learning the fundamental abstract of TensorFlow. It is based from Deep Learning nanodegree foundation in Udacity and updated to my own version.

Architecture

It uses a Python class to represent a generic node(Node). The nodes perform both their own calculations and those of input edges.

class Node(object):
    def __init__(self, inbound_nodes=[]):
        self.inbound_nodes = inbound_nodes  # Node(s) from which this Node receives values
        self.outbound_nodes = []   # Node(s) to which this Node passes values
        self.gradients = {}
        # For each inbound Node here, add this Node as an outbound Node to _that_ Node.
        for n in self.inbound_nodes:
            n.outbound_nodes.append(self)
        self.value = None  # A calculated value

    # Placeholder
    def forward(self):
        # Compute output value based on 'inbound_nodes' and store the result in self.value.
        raise NotImplemented  # Subclasses must implement this function to avoid errors.
    

    # Placeholder
    def backward(self):
        raise NotImplemented  # Subclasses must implement this function to avoid errors.

Graph

The topological_sort function creates graph of neural networks by implementing topological sorting using Kahn's Algorithm.

• Define the graph of nodes and edges.
• Propagation values through the graph.

How to use

Let's create simple neural network(simple_nn.py).

This model is not trained. Check the calculation of gradients with one back propagation.

• Set node into variable

X, W, b = Input(), Input(), Input()
y = Input()
l = Linear(X, W, b)
s = Sigmoid(l)
cost = MSE(y, s)

X_ = np.array([[3, 2], [-1, -2]])
W_ = np.array([[2], [3]])
b_ = np.array([-3])
y_ = np.array([[11], [2]])

feed_dict = {
    X: X_,
    W: W_,
    b: b_,
    y: y_
}

• Create graph with feed dict

graph = topological_sort(feed_dict)

• forward and backward(just one step)

forward_and_backward(graph)

• Calculate the gradients of \( W \) and \( b \)

print(W.gradients[W])
print(b.gradients[b])

Examples

Regression - Boston House Prices dataset

Classification - Mnist