Merge pull request cornucopia-rs#15 from LouisGariepy/feature-live-db

Feature live db

README.md

@@ -81,9 +81,11 @@ Aside from the dependencies, you will need the `cornucopia` cli to generate your
 ## Concepts
 This section explain a bit more about how cornucopia works. If you just want to get started, you should take a look at the [basic example](https://github.com/LouisGariepy/cornucopia/tree/main/examples/basic).
 
-Cornucopia is pretty simple to use. Your migrations and queries should each reside in a dedicated folder, and from there the CLI takes care of the rest for you. In the next sections, we'll explore the basic usage, but feel free to explore the CLI's whole interface using the `--help` option at any point.
+Cornucopia is pretty simple to use. Your migrations and queries should each reside in a dedicated folder, and from there the CLI takes care of the rest for you. In the next sections, we'll explore the basic usage, but feel free to explore the CLI's whole interface using the `--help` option at any point. For convenience, this is also available [in this repository](https://github.com/LouisGariepy/cornucopia/blob/main/cli.md).
 
 ### Migrations
+The basic `cornucopia generate` command spins a new container, runs your migrations, generates your queries and cleanups the container. If you want to manage the database and migrations yourself, use the `cornucopia generate live` command to connect to an arbitrary live database. Keep in mind that your queries must still be otherwise compatible with cornucopia (e.g. with regards to [supported types](https://github.com/LouisGariepy/cornucopia#supported-types) and [annotation syntax](https://github.com/LouisGariepy/cornucopia#query-annotation-syntax).
+
 New migrations can be added using the command `cornucopia migration new`. Cornucopia will automatically manage migrations when it generates your Rust modules, but you can also use the command `cornucopia migration run` to run migrations on your production database too if you so desire.
 
 ### Queries
@@ -129,7 +131,7 @@ pub async fn authors(client: &Client) -> Result<Vec<(i32, String, String)>, Erro
 ```
 Not bad! The generated function uses prepared statements, a statement cache, and strong typing (Notice how the returned rows' types have been inferred!). This is only a taste of what you can achieve, but should be fairly representative of what's going on under the hood.
 
-### Meta query syntax
+### Query annotation syntax
 As you may have noticed from the previous section, this little comment `--! authors()*` is doing a lot of heavy-lifting for us. It tells `cornucopia` to generate a function named `authors` with no parameters. Since there is no specified return, cornucopia will automatically infer what's being returned. Then, there's the asterisk `*` which signals that this query will return zero or more results. That's how we ended up with a `Vec` return in the generated query in the [section above](#generated-modules).
 
 Note that comments that do not start with `--!` (e.g. `-- This`) are simply ignored by `cornucopia`, so feel free to use them as you usually would.
@@ -150,7 +152,7 @@ The name of the generated function. It has to be a valid PostgresQL and Rust ide
 
 The parameters of the prepared statement, separated by commas (with an optional trailing comma.) 
 
-The order in which parameters are given corresponds to the parameter number (e.g. the first parameter is `$1` in the statement). **Every PostgreSQL parameter `$i` must have a corresponding parameter in the meta parameter list**.
+The order in which parameters are given corresponds to the parameter number (e.g. the first parameter is `$1` in the statement). **Every PostgreSQL parameter `$i` must have a corresponding parameter in the annotation parameter list**.
 
 #### Return type
 There are two kinds of returns, implicit and explicit. 

examples/basic/README.md

@@ -1,9 +1,9 @@
 # Example
 ## Before starting
-Please follow the the [install procedure](../../README.md#install). Before running CLI commands, we encourage you to explore what it can do and what options it offers via the `--help` flag.
+Please follow the the [install procedure](../../README.md#install). Before running CLI commands, we encourage you to explore its interface, either via the `--help` flag, or by taking a look at this [document](https://github.com/LouisGariepy/cornucopia/blob/main/cli.md).
 
 ## Take a look!
-This crate contains a fully working example. There are a few queries and migrations defined for you in the `migrations/` and `queries/` folders. The Rust modules have already been generated in the `src/cornucopia.rs` file. Finally, in `src/main.rs` you can see the queries in action, as you would use them in your own project. Taking a look should give you a solid idea of what `cornucopia` is about, enough to get you started in your own project. **Bear in mind that while it is instructive to look at this example, you will need a live database to actually execute the main file of this example since it tries to run the queries against an actual database.**. 
+This crate contains a fully working example. There are a few queries and migrations defined for you in the `migrations/` and `queries/` folders. The Rust modules have already been generated in the `src/cornucopia.rs` file. Finally, in `src/main.rs` you can see the queries in action, as you would use them in your own project. Taking a look should give you a solid idea of what `cornucopia` is about, enough to get you started in your own project. **Bear in mind that while it is instructive to look at this example, you will need a live database to execute the main file of this example since it will actually try to connect to a database and run the queries.**. 
 
 ## (Optional) Running the example
 If you want to be able to run this example, you should

src/cli.rs

@@ -10,19 +10,19 @@ pub struct Args {
 #[derive(Debug, Subcommand)]
 pub enum Action {
     /// Create and run migrations
-    Migration {
+    Migrations {
         #[clap(subcommand)]
-        action: MigrationAction,
+        action: MigrationsAction,
         /// Folder containing the migrations
-        #[clap(short, long, default_value = "migrations")]
+        #[clap(short, long, default_value = "migrations/")]
         migrations_path: String,
     },
     /// Generate Rust modules from queries
-    Generation {
+    Generate {
         /// Folder containing the migrations
         #[clap(short, long)]
         podman: bool,
-        /// Folder containing the migrations
+        /// Folder containing the migrations (ignored if using the `live` command)
         #[clap(short, long, default_value = "migrations/")]
         migrations_path: String,
         /// Folder containing the queries
@@ -31,23 +31,29 @@ pub enum Action {
         /// Destination folder for generated modules
         #[clap(short, long, default_value = "src/cornucopia.rs")]
         destination: String,
+        #[clap(subcommand)]
+        action: Option<GenerateLiveAction>,
     },
 }
 
 #[derive(Debug, Subcommand)]
-pub enum MigrationAction {
+pub enum MigrationsAction {
     /// Create a new migration
     New { name: String },
     /// Run all migrations
     Run {
         /// Postgres url to the database
         #[clap(long)]
-        user: String,
-        #[clap(long)]
-        password: String,
-        #[clap(long)]
-        host: String,
-        #[clap(long)]
-        port: u16,
+        url: String,
+    },
+}
+
+#[derive(Debug, Subcommand)]
+pub enum GenerateLiveAction {
+    /// Generate your modules against your own db
+    Live {
+        /// Postgres url to the database
+        #[clap(short, long)]
+        url: String,
     },
 }

src/error.rs

@@ -1,5 +1,6 @@
 use crate::codegen::error::Error as CodegenError;
 use crate::container::error::Error as ContainerError;
+use crate::pool::error::Error as PoolBuilderError;
 use crate::prepare_queries::error::Error as PrepareQueriesError;
 use crate::read_queries::error::Error as ReadQueriesError;
 use crate::run_migrations::error::Error as MigrationError;
@@ -17,6 +18,7 @@ pub enum Error {
     Migration(#[from] MigrationError),
     PoolCreation(#[from] CreatePoolError),
     Pool(#[from] deadpool_postgres::PoolError),
+    PoolBuilder(#[from] PoolBuilderError),
     FmtError(#[from] FmtError),
 }
 

src/integration/test.rs

@@ -16,10 +16,10 @@ use deadpool_postgres::Client;
 
 async fn setup() -> Result<Client, crate::error::Error> {
     use crate::run_migrations::run_migrations;
-    use crate::{container, pool::cli_pool};
+    use crate::{container, pool::cornucopia_pool};
 
     container::setup(true)?;
-    let pool = cli_pool()?;
+    let pool = cornucopia_pool()?;
     let client = pool.get().await?;
     run_migrations(&client, "src/integration/migrations").await?;
 

src/lib.rs

@@ -16,11 +16,11 @@ mod parse_file;
 mod sanitize;
 
 use clap::{Result, StructOpt};
-use cli::{Action, Args, MigrationAction};
+use cli::{Action, Args, GenerateLiveAction, MigrationsAction};
 use codegen::generate;
 use error::FmtError;
 use pg_type::TypeRegistrar;
-use pool::{cli_pool, create_pool};
+use pool::{cornucopia_pool, from_url};
 use prepare_queries::prepare_modules;
 use read_queries::read_queries;
 use run_migrations::run_migrations;
@@ -35,50 +35,56 @@ pub async fn run() -> Result<(), Error> {
     let args = Args::parse();
 
     match args.action {
-        Action::Migration {
+        Action::Migrations {
             action,
             migrations_path,
         } => match action {
-            MigrationAction::New { name } => {
+            MigrationsAction::New { name } => {
                 let unix_ts = OffsetDateTime::now_utc().unix_timestamp();
                 let file_path =
                     Path::new(&migrations_path).join(format!("{}_{}.sql", unix_ts, name));
                 std::fs::write(file_path, "-- Write your migration SQL here\n")?;
                 Ok(())
             }
-            MigrationAction::Run {
-                user,
-                password,
-                host,
-                port,
-            } => {
-                let client = create_pool(user, password, host, port)?.get().await?;
+            MigrationsAction::Run { url } => {
+                let client = pool::from_url(&url)?.get().await?;
                 run_migrations(&client, &migrations_path).await?;
 
                 Ok(())
             }
         },
-        Action::Generation {
+        Action::Generate {
+            action,
             podman,
             migrations_path,
             queries_path,
             destination,
         } => {
             let mut type_registrar = TypeRegistrar::default();
-            if let Err(e) = generation(
-                &mut type_registrar,
-                podman,
-                migrations_path,
-                queries_path,
-                destination,
-            )
-            .await
-            {
-                container::cleanup(podman)?;
-                return Err(e);
-            }
+            match action {
+                Some(GenerateLiveAction::Live { url }) => {
+                    let modules = read_queries(&queries_path)?;
+                    let client = from_url(&url)?.get().await?;
+                    let modules = prepare_modules(&client, &mut type_registrar, modules).await?;
+                    generate(&type_registrar, modules, &destination)?;
+                }
+                None => {
+                    if let Err(e) = generate_action(
+                        &mut type_registrar,
+                        podman,
+                        migrations_path,
+                        queries_path,
+                        destination,
+                    )
+                    .await
+                    {
+                        container::cleanup(podman)?;
+                        return Err(e);
+                    }
 
-            format_project()?;
+                    format_project()?;
+                }
+            }
 
             Ok(())
         }
@@ -93,7 +99,7 @@ pub(crate) fn format_project() -> Result<(), FmtError> {
     }
 }
 
-pub(crate) async fn generation(
+pub(crate) async fn generate_action(
     type_registrar: &mut TypeRegistrar,
     podman: bool,
     migrations_path: String,
@@ -102,10 +108,10 @@ pub(crate) async fn generation(
 ) -> Result<(), Error> {
     let modules = read_queries(&queries_path)?;
     container::setup(podman)?;
-    let client = cli_pool()?.get().await?;
+    let client = cornucopia_pool()?.get().await?;
     run_migrations(&client, &migrations_path).await?;
-    let modules = prepare_modules(&client, type_registrar, modules).await?;
-    generate(type_registrar, modules, &destination)?;
+    let prepared_modules = prepare_modules(&client, type_registrar, modules).await?;
+    generate(type_registrar, prepared_modules, &destination)?;
     container::cleanup(podman)?;
 
     Ok(())

src/pool.rs

@@ -1,26 +1,39 @@
-use deadpool_postgres::{Config, CreatePoolError, Pool, Runtime};
+use error::Error;
+use std::str::FromStr;
+
+use deadpool_postgres::{Config, Pool, Runtime};
 use tokio_postgres::NoTls;
 
-pub fn cli_pool() -> Result<Pool, CreatePoolError> {
-    create_pool(
-        String::from("postgres"),
-        String::from("postgres"),
-        String::from("127.0.0.1"),
-        5432,
-    )
+pub fn from_url(url: &str) -> Result<Pool, Error> {
+    let config = tokio_postgres::Config::from_str(url)?;
+    let manager = deadpool_postgres::Manager::new(config, tokio_postgres::NoTls);
+    let pool = deadpool_postgres::Pool::builder(manager).build()?;
+    Ok(pool)
 }
 
-pub fn create_pool(
-    user: String,
-    password: String,
-    host: String,
-    port: u16,
-) -> Result<Pool, CreatePoolError> {
+pub fn cornucopia_pool() -> Result<Pool, Error> {
     let mut cfg = Config::new();
-    cfg.user = Some(user);
-    cfg.password = Some(password);
-    cfg.host = Some(host);
-    cfg.port = Some(port);
+    cfg.user = Some(String::from("postgres"));
+    cfg.password = Some(String::from("postgres"));
+    cfg.host = Some(String::from("127.0.0.1"));
+    cfg.port = Some(5432);
     cfg.dbname = Some(String::from("postgres"));
-    cfg.create_pool(Some(Runtime::Tokio1), NoTls)
+    let pool = cfg.create_pool(Some(Runtime::Tokio1), NoTls)?;
+    Ok(pool)
+}
+
+pub mod error {
+    use deadpool_postgres::BuildError as PoolBuilderError;
+    use deadpool_postgres::CreatePoolError;
+    use thiserror::Error as ThisError;
+    #[derive(Debug, ThisError)]
+    #[error("An error happened when trying to acquire a connection")]
+    pub enum Error {
+        #[error("Invalid database URL")]
+        PoolBuilder(#[from] PoolBuilderError),
+        #[error("Invalid database URL")]
+        DbUrl(#[from] CreatePoolError),
+        #[error("Invalid database URL")]
+        Db(#[from] tokio_postgres::Error),
+    }
 }