Skip to content

Commit

Permalink
small changes in guide
Browse files Browse the repository at this point in the history
  • Loading branch information
Sigrid Keydana committed Sep 27, 2018
1 parent e5964df commit 024c612
Show file tree
Hide file tree
Showing 2 changed files with 43 additions and 26 deletions.
65 changes: 42 additions & 23 deletions vignettes/eager_guide.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ menu:
name: "Keras with eager execution"
identifier: "keras-eager"
parent: "keras-using-keras"
weight: 10
weight: 15

---

Expand Down Expand Up @@ -98,12 +98,18 @@ model <- iris_regression_model()
```

At this point, the shapes of the model's weights are still unknown (note how no `input_shape` has been defined for its first layer).
You can, however, already call the model on some test data:
You can, however, already call the model on some dummy data:

```{r}
model(k_constant(matrix(1:6, nrow = 2, ncol = 3)))
```

```
tf.Tensor(
[[-1.1474639]
[-1.0472134]], shape=(2, 1), dtype=float32)
```

After that call, you can inspect the model's weights using

```{r}
Expand All @@ -114,14 +120,11 @@ This will not just display the tensor shapes, but the actual weight values.

## Losses and optimizers


> JJA: We should explain why we need to use the tf loss and optimizer here
An appropriate loss function for a regression task like this is mean squared error:

```{r}
mse_loss <- function(y_true, y_pred, x) {
# can't use Keras loss_mean_squared_error() here
# it's required to use a TensorFlow function here, not loss_mean_squared_error() from Keras
mse <- tf$losses$mean_squared_error(y_true, y_pred)
# here you could compute and add other losses
mse
Expand All @@ -131,7 +134,7 @@ mse_loss <- function(y_true, y_pred, x) {
Note how we have to use loss functions from TensorFlow, not the Keras equivalents. In the same vein, we need to use an optimizer from the `tf$train` module.

```{r}
# can't use Keras' optimizer_adam()
# have to use an optimizer from tf$train, not Keras
optimizer <- tf$train$AdamOptimizer()
```

Expand Down Expand Up @@ -182,8 +185,33 @@ Datasets are available in non-eager (graph) execution as well. However, in eager
batch
```

> JJA: show output for simple data inspection (another case up above)
```
[[1]]
tf.Tensor(
[[1.4 5.1 0.2]
[1.4 4.9 0.2]
[1.3 4.7 0.2]
[1.5 4.6 0.2]
[1.4 5. 0.2]
[1.7 5.4 0.4]
[1.4 4.6 0.3]
[1.5 5. 0.2]
[1.4 4.4 0.2]
[1.5 4.9 0.1]], shape=(10, 3), dtype=float32)
[[2]]
tf.Tensor(
[[3.5]
[3. ]
[3.2]
[3.1]
[3.6]
[3.9]
[3.4]
[3.4]
[2.9]
[3.1]], shape=(10, 1), dtype=float32)
```

## Training loop

Expand Down Expand Up @@ -212,7 +240,7 @@ for (i in seq_len(n_epochs)) {
})
cat("Total loss (epoch): ", i, ": ", total_loss$numpy(), "\n")
cat("Total loss (epoch): ", i, ": ", as.numeric(total_loss), "\n")
}
```

Expand Down Expand Up @@ -284,8 +312,6 @@ Getting predictions on the test set is just a call to `model`, just like trainin
model(x_test)
```

> JJA: show output?

## Saving and restoring model weights

Expand Down Expand Up @@ -317,14 +343,11 @@ This call saves model weights only, not the complete graph. Thus on restore, you
checkpoint$restore(tf$train$latest_checkpoint(checkpoint_dir))
```

To get predictions from a freshly restored model, you need to pass in the test data as a tensor.
Two ways to achieve this are shown below: using `k_constant` on `x_test` directly or using an iterator, like we do in the training loop.
You can then obtain predictions from the restored model, on the test set as a whole or batch-wise, using an iterator.

```{r}
# way 1
model(k_constant(x_test))
model(x_test)
# way 2
iter <- make_iterator_one_shot(test_dataset)
until_out_of_range({
batch <- iterator_get_next(iter)
Expand All @@ -339,8 +362,6 @@ until_out_of_range({
Here is the complete example.

```{r}
reticulate::use_condaenv("tf-10-gpu", required = TRUE)
library(keras)
use_implementation("tensorflow")
Expand Down Expand Up @@ -441,7 +462,7 @@ if (!restore) {
})
cat("Total loss (epoch): ", i, ": ", total_loss$numpy(), "\n")
cat("Total loss (epoch): ", i, ": ", as.numeric(total_loss), "\n")
checkpoint$save(file_prefix = checkpoint_prefix)
}
Expand All @@ -452,10 +473,8 @@ if (!restore) {
# Get model predictions on test set ---------------------------------------
# way 1
model(k_constant(x_test))
model(x_test)
# way 2
iter <- make_iterator_one_shot(test_dataset)
until_out_of_range({
batch <- iterator_get_next(iter)
Expand Down
4 changes: 1 addition & 3 deletions vignettes/guide_keras.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -491,7 +491,7 @@ model <- load_model_hdf5('my_model.h5')

## Eager execution

TensorFlow [Eager execution](https://www.tensorflow.org/guide/eager) is an imperative programming environment that evaluates operations immediately. This is not required for Keras,
[Eager execution](eager_guide.Rmd) is an imperative programming environment that evaluates operations immediately. This is not required for Keras,
but is supported by the TensorFlow backend and useful for inspecting your program
and debugging.

Expand All @@ -506,8 +506,6 @@ use_implementation("tensorflow")
When using the TensorFlow implementation, all model-building APIs are compatible with eager
execution.

Consult the [RStudio TensorFlow blog](https://blogs.rstudio.com/tensorflow/) for examples
using eager execution.

## Distribution

Expand Down

0 comments on commit 024c612

Please sign in to comment.