Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for generic timeline #525

Merged
merged 58 commits into from
Oct 7, 2019
Merged
Show file tree
Hide file tree
Changes from 52 commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
2112288
Begin adding support for timeline
trinity-1686a Apr 11, 2019
ff75bf4
fix some bugs with parser
trinity-1686a Apr 11, 2019
9e7b060
fmt
trinity-1686a Apr 11, 2019
6a32146
add error reporting for parser
trinity-1686a Apr 11, 2019
9902893
add tests for timeline query parser
trinity-1686a Apr 11, 2019
c009cba
add rejection tests for parse
trinity-1686a Apr 12, 2019
f55113f
begin adding support for lists
trinity-1686a Apr 13, 2019
4226295
add sqlite migration
trinity-1686a Apr 14, 2019
9d8b4da
end adding lists
trinity-1686a Apr 14, 2019
5b2f046
cargo fmt
trinity-1686a Apr 15, 2019
701ec1f
try to add some tests
trinity-1686a Apr 16, 2019
2086ce7
Add some constraint to db, and fix list test
trinity-1686a Apr 17, 2019
51e5760
add more tests for lists
trinity-1686a Apr 17, 2019
2e08029
add support for lists in query executor
trinity-1686a Apr 17, 2019
7a92b4f
Merge branch 'master' into timeline
trinity-1686a Apr 23, 2019
65c47c2
Merge branch 'master' into timeline
trinity-1686a Apr 23, 2019
b38a1c1
add keywords for including/excluding boosts and likes
trinity-1686a Apr 25, 2019
b3ff040
cargo fmt
trinity-1686a Apr 25, 2019
d006de8
add function to list lists used by query
trinity-1686a Apr 25, 2019
0967181
add lang support
trinity-1686a Apr 25, 2019
afc17f1
add timeline creation error message when using unexisting lists
trinity-1686a Apr 25, 2019
1f70caf
Merge branch 'master' into timeline
elegaanz Apr 28, 2019
eec8b34
Update .po files
elegaanz Apr 28, 2019
a15633f
WIP: interface for timelines
elegaanz Apr 28, 2019
53dd1e6
Merge branch 'master' into timeline
trinity-1686a Apr 29, 2019
c96e53e
don't use diesel for migrations
trinity-1686a Apr 29, 2019
cd73f4f
add some tests for timeline
trinity-1686a May 1, 2019
7eb1d5c
cargo fmt
trinity-1686a May 1, 2019
fd9f316
remove timeline order
trinity-1686a May 1, 2019
2c0ef71
Merge branch 'master' into timeline
trinity-1686a May 4, 2019
457f832
fix tests
trinity-1686a May 5, 2019
22747f8
add tests for timeline creation failure
trinity-1686a May 10, 2019
5280c1d
cargo fmt
trinity-1686a May 10, 2019
4dafdb8
Merge branch 'master' into timeline
trinity-1686a May 10, 2019
cf165c4
Merge branch 'master' into timeline
trinity-1686a May 18, 2019
aad1609
add tests for timelines
trinity-1686a May 19, 2019
0f74891
add test for matching direct lists and keywords
trinity-1686a May 19, 2019
08cefa8
add test for language filtering
trinity-1686a May 26, 2019
87f6637
Merge branch 'master' into timeline
trinity-1686a Jun 15, 2019
caaec07
Merge branch 'timeline' of github.com:Plume-org/Plume into timeline
elegaanz Jun 22, 2019
72bc84a
Add a more complex test for Timeline::matches, and fix TQ::matches fo…
elegaanz Jun 22, 2019
7957de2
Make the main crate compile + FMT
elegaanz Jun 22, 2019
aa521b5
Use the new timeline system
elegaanz Jun 24, 2019
91cf1b5
Cargo fmt
elegaanz Jun 24, 2019
aea9f99
Try to fix the migration
elegaanz Jun 24, 2019
6987b8f
Fix tests
elegaanz Jun 24, 2019
cb6c0a6
Fix the test (for real this time ?)
elegaanz Jun 24, 2019
5ee87ef
Fix the tests ? + fmt
elegaanz Jun 25, 2019
e1a225a
Use Kind::Like and Kind::Reshare when needed
elegaanz Jul 7, 2019
07d788b
Forgot to run cargo fmt once again
elegaanz Jul 7, 2019
fdd4132
revert translations
trinity-1686a Jul 23, 2019
e746b59
Merge branch 'master' into timeline
trinity-1686a Jul 23, 2019
98904c9
fix reviewed stuff
trinity-1686a Jul 28, 2019
12a952f
reduce code duplication by macros
trinity-1686a Aug 13, 2019
e03e3fa
cargo fmt
trinity-1686a Aug 13, 2019
29d2815
Merge branch 'master' into timeline
trinity-1686a Sep 13, 2019
7e0f967
Merge remote-tracking branch 'origin/master' into timeline
trinity-1686a Sep 13, 2019
bb84e20
Merge branch 'master' into timeline
igalic Sep 18, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions build.rs
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
extern crate rsass;
extern crate ructe;
use ructe::*;
use ructe::Ructe;
use std::process::{Command, Stdio};
use std::{env, fs::*, io::Write, path::PathBuf};
use std::{fs::*, io::Write};

fn compute_static_hash() -> String {
//"find static/ -type f ! -path 'static/media/*' | sort | xargs stat -c'%n %Y' | openssl dgst -r"
Expand Down Expand Up @@ -37,9 +37,10 @@ fn compute_static_hash() -> String {
}

fn main() {
let out_dir = PathBuf::from(env::var("OUT_DIR").unwrap());
let in_dir = PathBuf::from(env::var("CARGO_MANIFEST_DIR").unwrap()).join("templates");
compile_templates(&in_dir, &out_dir).expect("compile templates");
Ructe::from_env()
.expect("This must be run with cargo")
.compile_templates("templates")
trinity-1686a marked this conversation as resolved.
Show resolved Hide resolved
.expect("compile templates");

println!("cargo:rerun-if-changed=static/css/_article.scss");
println!("cargo:rerun-if-changed=static/css/_forms.scss");
Expand Down
6 changes: 6 additions & 0 deletions migrations/postgres/2019-04-30-145757_timeline/down.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
-- This file should undo anything in `up.sql`

DROP TABLE timeline;
DROP TABLE timeline_definition;
DROP TABLE list_elems;
DROP TABLE lists;
31 changes: 31 additions & 0 deletions migrations/postgres/2019-04-30-145757_timeline/up.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
-- Your SQL goes here

CREATE TABLE timeline_definition(
id SERIAL PRIMARY KEY,
user_id integer REFERENCES users ON DELETE CASCADE,
name VARCHAR NOT NULL,
query VARCHAR NOT NULL,
CONSTRAINT timeline_unique_user_name UNIQUE(user_id, name)
);

CREATE TABLE timeline(
id SERIAL PRIMARY KEY,
post_id integer NOT NULL REFERENCES posts ON DELETE CASCADE,
timeline_id integer NOT NULL REFERENCES timeline_definition ON DELETE CASCADE
);

CREATE TABLE lists(
id SERIAL PRIMARY KEY,
name VARCHAR NOT NULL,
user_id integer REFERENCES users ON DELETE CASCADE,
type integer NOT NULL,
CONSTRAINT list_unique_user_name UNIQUE(user_id, name)
);

CREATE TABLE list_elems(
id SERIAL PRIMARY KEY,
list_id integer NOT NULL REFERENCES lists ON DELETE CASCADE,
user_id integer REFERENCES users ON DELETE CASCADE,
blog_id integer REFERENCES blogs ON DELETE CASCADE,
word VARCHAR
);
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
-- This file should undo anything in `up.sql`
DELETE FROM timeline_definition WHERE name = 'Your feed';
DELETE FROM timeline_definition WHERE name = 'Local feed' AND query = 'local';
DELETE FROM timeline_definition WHERE name = 'Federared feed' AND query = 'all';
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
-- Your SQL goes here
--#!|conn: &Connection, path: &Path| {
--#! let mut i = 0;
trinity-1686a marked this conversation as resolved.
Show resolved Hide resolved
--#! loop {
--#! if let Ok(users) = super::users::User::get_local_page(conn, (i * 20, (i + 1) * 20)) {
--#! if users.is_empty() {
--#! break;
--#! }
--#!
--#! for u in users {
--#! super::timeline::Timeline::new_for_user(conn, u.id, "Your feed".into(), format!("followed or author in [ {} ]", u.fqn)).expect("User feed creation error");
--#! }
--#! i += 1;
--#! } else {
--#! break;
--#! }
--#! }
--#!
--#! super::timeline::Timeline::new_for_instance(conn, "Local feed".into(), "local".into()).expect("Local feed creation error");
--#! super::timeline::Timeline::new_for_instance(conn, "Federated feed".into(), "all".into()).expect("Federated feed creation error");
--#! Ok(())
--#!}
6 changes: 6 additions & 0 deletions migrations/sqlite/2019-04-30-145757_timeline/down.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
-- This file should undo anything in `up.sql`

DROP TABLE timeline;
DROP TABLE timeline_definition;
DROP TABLE list_elems;
DROP TABLE lists;
31 changes: 31 additions & 0 deletions migrations/sqlite/2019-04-30-145757_timeline/up.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
-- Your SQL goes here

CREATE TABLE timeline_definition(
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
user_id INTEGER REFERENCES users(id) ON DELETE CASCADE,
name VARCHAR NOT NULL,
query VARCHAR NOT NULL,
CONSTRAINT timeline_unique_user_name UNIQUE(user_id, name)
);

CREATE TABLE timeline(
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
post_id integer NOT NULL REFERENCES posts(id) ON DELETE CASCADE,
timeline_id integer NOT NULL REFERENCES timeline_definition(id) ON DELETE CASCADE
);

CREATE TABLE lists(
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
name VARCHAR NOT NULL,
user_id integer REFERENCES users(id) ON DELETE CASCADE,
type integer NOT NULL,
CONSTRAINT timeline_unique_user_name UNIQUE(user_id, name)
);

CREATE TABLE list_elems(
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
list_id integer NOT NULL REFERENCES lists(id) ON DELETE CASCADE,
user_id integer REFERENCES users(id) ON DELETE CASCADE,
blog_id integer REFERENCES blogs(id) ON DELETE CASCADE,
word VARCHAR
);
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
-- This file should undo anything in `up.sql`
DELETE FROM timeline_definition WHERE name = 'Your feed';
DELETE FROM timeline_definition WHERE name = 'Local feed' AND query = 'local';
DELETE FROM timeline_definition WHERE name = 'Federared feed' AND query = 'all';
igalic marked this conversation as resolved.
Show resolved Hide resolved
22 changes: 22 additions & 0 deletions migrations/sqlite/2019-06-24-105533_use_timelines_for_feed/up.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
-- Your SQL goes here
--#!|conn: &Connection, path: &Path| {
--#! let mut i = 0;
--#! loop {
trinity-1686a marked this conversation as resolved.
Show resolved Hide resolved
--#! if let Ok(users) = super::users::User::get_local_page(conn, (i * 20, (i + 1) * 20)) {
--#! if users.is_empty() {
--#! break;
--#! }
--#!
--#! for u in users {
--#! super::timeline::Timeline::new_for_user(conn, u.id, "Your feed".into(), format!("followed or author in [ {} ]", u.fqn)).expect("User feed creation error");
--#! }
--#! i += 1;
--#! } else {
--#! break;
--#! }
--#! }
--#!
--#! super::timeline::Timeline::new_for_instance(conn, "Local feed".into(), "local".into()).expect("Local feed creation error");
--#! super::timeline::Timeline::new_for_instance(conn, "Federated feed".into(), "all".into()).expect("Federated feed creation error");
--#! Ok(())
--#!}
28 changes: 10 additions & 18 deletions plume-models/src/blogs.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use activitypub::{actor::Group, collection::OrderedCollection, object::Image, CustomObject};
use chrono::NaiveDateTime;
use diesel::{self, ExpressionMethods, QueryDsl, RunQueryDsl, SaveChangesDsl};
use diesel::{self, ExpressionMethods, OptionalExtension, QueryDsl, RunQueryDsl, SaveChangesDsl};
use openssl::{
hash::MessageDigest,
pkey::{PKey, Private},
Expand Down Expand Up @@ -133,10 +133,8 @@ impl Blog {
pub fn find_by_fqn(c: &PlumeRocket, fqn: &str) -> Result<Blog> {
let from_db = blogs::table
.filter(blogs::fqn.eq(fqn))
.limit(1)
.load::<Blog>(&*c.conn)?
.into_iter()
.next();
.first(&*c.conn)
.optional()?;
igalic marked this conversation as resolved.
Show resolved Hide resolved
if let Some(from_db) = from_db {
Ok(from_db)
} else {
Expand Down Expand Up @@ -569,9 +567,8 @@ pub(crate) mod tests {
Instance::get_local().unwrap().id
);
// TODO add tests for remote instance

Ok(())
});
})
}

#[test]
Expand Down Expand Up @@ -671,9 +668,8 @@ pub(crate) mod tests {
.unwrap()
.iter()
.any(|b| b.id == blog[1].id));

Ok(())
});
})
}

#[test]
Expand All @@ -696,9 +692,8 @@ pub(crate) mod tests {
.unwrap();

assert_eq!(Blog::find_by_fqn(&r, "SomeName").unwrap().id, blog.id);

Ok(())
});
})
}

#[test]
Expand All @@ -720,9 +715,8 @@ pub(crate) mod tests {
.unwrap();

assert_eq!(blog.fqn, "SomeName");

Ok(())
});
})
}

#[test]
Expand All @@ -733,9 +727,8 @@ pub(crate) mod tests {

blogs[0].delete(conn, &get_searcher()).unwrap();
assert!(Blog::get(conn, blogs[0].id).is_err());

Ok(())
});
})
}

#[test]
Expand Down Expand Up @@ -804,9 +797,8 @@ pub(crate) mod tests {
assert!(Blog::get(conn, blog[1].id).is_err());
user[1].delete(conn, &searcher).unwrap();
assert!(Blog::get(conn, blog[0].id).is_err());

Ok(())
});
})
}

#[test]
Expand Down Expand Up @@ -867,6 +859,6 @@ pub(crate) mod tests {
assert_eq!(blog.banner_url(conn), blogs[0].banner_url(conn));

Ok(())
});
})
}
}
3 changes: 1 addition & 2 deletions plume-models/src/comments.rs
Original file line number Diff line number Diff line change
Expand Up @@ -436,8 +436,7 @@ mod tests {
}
_ => panic!("Unexpected result"),
};

Ok(())
});
})
}
}
16 changes: 16 additions & 0 deletions plume-models/src/db_conn.rs
Original file line number Diff line number Diff line change
Expand Up @@ -59,3 +59,19 @@ impl CustomizeConnection<Connection, ConnError> for PragmaForeignKey {
})
}
}

#[cfg(test)]
pub(crate) mod tests {
use super::*;
use diesel::Connection as _;

#[derive(Debug)]
pub struct TestConnectionCustomizer;
impl CustomizeConnection<Connection, ConnError> for TestConnectionCustomizer {
fn on_acquire(&self, conn: &mut Connection) -> Result<(), ConnError> {
PragmaForeignKey.on_acquire(conn)?;
Ok(conn.begin_test_transaction().unwrap())
}
}

}
3 changes: 2 additions & 1 deletion plume-models/src/follows.rs
Original file line number Diff line number Diff line change
Expand Up @@ -235,7 +235,8 @@ mod tests {
)
.expect("Couldn't insert new follow");
assert_eq!(follow.ap_url, String::from("https://some.url/"));

Ok(())
});
})
}
}
13 changes: 1 addition & 12 deletions plume-models/src/inbox.rs
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,6 @@ pub(crate) mod tests {
}
_ => panic!("Unexpected result"),
};

Ok(())
});
}
Expand Down Expand Up @@ -170,7 +169,6 @@ pub(crate) mod tests {
}
_ => panic!("Unexpected result"),
};

Ok(())
});
}
Expand Down Expand Up @@ -212,7 +210,6 @@ pub(crate) mod tests {
}
_ => panic!("Unexpected result"),
};

Ok(())
});
}
Expand Down Expand Up @@ -255,9 +252,8 @@ pub(crate) mod tests {
"type": "Delete",
});
assert!(super::inbox(&r, ok_act).is_ok());

Ok(())
});
})
}

#[test]
Expand All @@ -282,7 +278,6 @@ pub(crate) mod tests {
"type": "Delete",
});
assert!(super::inbox(&r, ok_act).is_ok());

Ok(())
});
}
Expand Down Expand Up @@ -336,7 +331,6 @@ pub(crate) mod tests {
}
_ => panic!("Unexpected result"),
}

Ok(())
});
}
Expand All @@ -362,7 +356,6 @@ pub(crate) mod tests {
}
_ => panic!("Unexpected result"),
}

Ok(())
});
}
Expand Down Expand Up @@ -401,7 +394,6 @@ pub(crate) mod tests {
"type": "Undo",
});
assert!(super::inbox(&r, ok_act).is_ok());

Ok(())
});
}
Expand Down Expand Up @@ -440,7 +432,6 @@ pub(crate) mod tests {
"type": "Undo",
});
assert!(super::inbox(&r, ok_act).is_ok());

Ok(())
});
}
Expand Down Expand Up @@ -479,7 +470,6 @@ pub(crate) mod tests {
"type": "Undo",
});
assert!(super::inbox(&r, ok_act).is_ok());

Ok(())
});
}
Expand Down Expand Up @@ -509,7 +499,6 @@ pub(crate) mod tests {
});

super::inbox(&r, act).unwrap();

Ok(())
});
}
Expand Down
Loading