Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for generic timeline #525

Merged
merged 58 commits into from
Oct 7, 2019
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
2112288
Begin adding support for timeline
trinity-1686a Apr 11, 2019
ff75bf4
fix some bugs with parser
trinity-1686a Apr 11, 2019
9e7b060
fmt
trinity-1686a Apr 11, 2019
6a32146
add error reporting for parser
trinity-1686a Apr 11, 2019
9902893
add tests for timeline query parser
trinity-1686a Apr 11, 2019
c009cba
add rejection tests for parse
trinity-1686a Apr 12, 2019
f55113f
begin adding support for lists
trinity-1686a Apr 13, 2019
4226295
add sqlite migration
trinity-1686a Apr 14, 2019
9d8b4da
end adding lists
trinity-1686a Apr 14, 2019
5b2f046
cargo fmt
trinity-1686a Apr 15, 2019
701ec1f
try to add some tests
trinity-1686a Apr 16, 2019
2086ce7
Add some constraint to db, and fix list test
trinity-1686a Apr 17, 2019
51e5760
add more tests for lists
trinity-1686a Apr 17, 2019
2e08029
add support for lists in query executor
trinity-1686a Apr 17, 2019
7a92b4f
Merge branch 'master' into timeline
trinity-1686a Apr 23, 2019
65c47c2
Merge branch 'master' into timeline
trinity-1686a Apr 23, 2019
b38a1c1
add keywords for including/excluding boosts and likes
trinity-1686a Apr 25, 2019
b3ff040
cargo fmt
trinity-1686a Apr 25, 2019
d006de8
add function to list lists used by query
trinity-1686a Apr 25, 2019
0967181
add lang support
trinity-1686a Apr 25, 2019
afc17f1
add timeline creation error message when using unexisting lists
trinity-1686a Apr 25, 2019
1f70caf
Merge branch 'master' into timeline
elegaanz Apr 28, 2019
eec8b34
Update .po files
elegaanz Apr 28, 2019
a15633f
WIP: interface for timelines
elegaanz Apr 28, 2019
53dd1e6
Merge branch 'master' into timeline
trinity-1686a Apr 29, 2019
c96e53e
don't use diesel for migrations
trinity-1686a Apr 29, 2019
cd73f4f
add some tests for timeline
trinity-1686a May 1, 2019
7eb1d5c
cargo fmt
trinity-1686a May 1, 2019
fd9f316
remove timeline order
trinity-1686a May 1, 2019
2c0ef71
Merge branch 'master' into timeline
trinity-1686a May 4, 2019
457f832
fix tests
trinity-1686a May 5, 2019
22747f8
add tests for timeline creation failure
trinity-1686a May 10, 2019
5280c1d
cargo fmt
trinity-1686a May 10, 2019
4dafdb8
Merge branch 'master' into timeline
trinity-1686a May 10, 2019
cf165c4
Merge branch 'master' into timeline
trinity-1686a May 18, 2019
aad1609
add tests for timelines
trinity-1686a May 19, 2019
0f74891
add test for matching direct lists and keywords
trinity-1686a May 19, 2019
08cefa8
add test for language filtering
trinity-1686a May 26, 2019
87f6637
Merge branch 'master' into timeline
trinity-1686a Jun 15, 2019
caaec07
Merge branch 'timeline' of github.com:Plume-org/Plume into timeline
elegaanz Jun 22, 2019
72bc84a
Add a more complex test for Timeline::matches, and fix TQ::matches fo…
elegaanz Jun 22, 2019
7957de2
Make the main crate compile + FMT
elegaanz Jun 22, 2019
aa521b5
Use the new timeline system
elegaanz Jun 24, 2019
91cf1b5
Cargo fmt
elegaanz Jun 24, 2019
aea9f99
Try to fix the migration
elegaanz Jun 24, 2019
6987b8f
Fix tests
elegaanz Jun 24, 2019
cb6c0a6
Fix the test (for real this time ?)
elegaanz Jun 24, 2019
5ee87ef
Fix the tests ? + fmt
elegaanz Jun 25, 2019
e1a225a
Use Kind::Like and Kind::Reshare when needed
elegaanz Jul 7, 2019
07d788b
Forgot to run cargo fmt once again
elegaanz Jul 7, 2019
fdd4132
revert translations
trinity-1686a Jul 23, 2019
e746b59
Merge branch 'master' into timeline
trinity-1686a Jul 23, 2019
98904c9
fix reviewed stuff
trinity-1686a Jul 28, 2019
12a952f
reduce code duplication by macros
trinity-1686a Aug 13, 2019
e03e3fa
cargo fmt
trinity-1686a Aug 13, 2019
29d2815
Merge branch 'master' into timeline
trinity-1686a Sep 13, 2019
7e0f967
Merge remote-tracking branch 'origin/master' into timeline
trinity-1686a Sep 13, 2019
bb84e20
Merge branch 'master' into timeline
igalic Sep 18, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
-- This file should undo anything in `up.sql`
DELETE FROM timeline_definition WHERE name = 'Your feed';
DELETE FROM timeline_definition WHERE name = 'Local feed' AND query = 'local';
DELETE FROM timeline_definition WHERE name = 'Federared feed' AND query = 'all';
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
-- Your SQL goes here
--#!|conn: &Connection, path: &Path| {
--#! let mut i = 0;
trinity-1686a marked this conversation as resolved.
Show resolved Hide resolved
--#! loop {
--#! let users = super::users::User::get_local_page(conn, (i * 20, (i + 1) * 20)).unwrap();
--#!
--#! if users.is_empty() {
--#! break;
--#! }
--#!
--#! for u in users {
--#! super::timeline::Timeline::new_for_user(conn, u.id, "Your feed".into(), format!("followed or author in [ {} ]", u.fqn)).unwrap();
--#! }
--#! i += 1;
--#!
--#! }
--#!
--#! super::timeline::Timeline::new_for_instance(conn, "Local feed".into(), "local".into()).expect("Local feed creation error");
--#! super::timeline::Timeline::new_for_instance(conn, "Federated feed".into(), "all".into()).expect("Federated feed creation error");
--#! Ok(())
--#!}
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
-- This file should undo anything in `up.sql`
DELETE FROM timeline_definition WHERE name = 'Your feed';
DELETE FROM timeline_definition WHERE name = 'Local feed' AND query = 'local';
DELETE FROM timeline_definition WHERE name = 'Federared feed' AND query = 'all';
igalic marked this conversation as resolved.
Show resolved Hide resolved
21 changes: 21 additions & 0 deletions migrations/sqlite/2019-06-24-105533_use_timelines_for_feed/up.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
-- Your SQL goes here
--#!|conn: &Connection, path: &Path| {
--#! let mut i = 0;
--#! loop {
trinity-1686a marked this conversation as resolved.
Show resolved Hide resolved
--#! let users = super::users::User::get_local_page(conn, (i * 20, (i + 1) * 20)).unwrap();
--#!
--#! if users.is_empty() {
--#! break;
--#! }
--#!
--#! for u in users {
--#! super::timeline::Timeline::new_for_user(conn, u.id, "Your feed".into(), format!("followed or author in [ {} ]", u.fqn)).unwrap();
--#! }
--#! i += 1;
--#!
--#! }
--#!
--#! super::timeline::Timeline::new_for_instance(conn, "Local feed".into(), "local".into()).expect("Local feed creation error");
--#! super::timeline::Timeline::new_for_instance(conn, "Federated feed".into(), "all".into()).expect("Federated feed creation error");
--#! Ok(())
--#!}
3 changes: 3 additions & 0 deletions plume-models/src/likes.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ use plume_common::activity_pub::{
};
use posts::Post;
use schema::likes;
use timeline::*;
use users::User;
use {Connection, Error, PlumeRocket, Result};

Expand Down Expand Up @@ -99,6 +100,8 @@ impl AsObject<User, activity::Like, &PlumeRocket> for Post {
},
)?;
res.notify(&c.conn)?;

Timeline::add_to_all_timelines(c, &self, Kind::Original)?;
trinity-1686a marked this conversation as resolved.
Show resolved Hide resolved
Ok(res)
}
}
Expand Down
67 changes: 4 additions & 63 deletions plume-models/src/posts.rs
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ use safe_string::SafeString;
use schema::posts;
use search::Searcher;
use tags::*;
use timeline::*;
use users::User;
use {ap_url, Connection, Error, PlumeRocket, Result, CONFIG};

Expand Down Expand Up @@ -182,15 +183,6 @@ impl Post {
query.get_results::<Post>(conn).map_err(Error::from)
}

pub fn get_recents(conn: &Connection, limit: i64) -> Result<Vec<Post>> {
posts::table
.order(posts::creation_date.desc())
.filter(posts::published.eq(true))
.limit(limit)
.load::<Post>(conn)
.map_err(Error::from)
}

pub fn get_recents_for_author(
conn: &Connection,
author: &User,
Expand Down Expand Up @@ -246,60 +238,6 @@ impl Post {
.map_err(Error::from)
}

/// Give a page of all the recent posts known to this instance (= federated timeline)
pub fn get_recents_page(conn: &Connection, (min, max): (i32, i32)) -> Result<Vec<Post>> {
posts::table
.order(posts::creation_date.desc())
.filter(posts::published.eq(true))
.offset(min.into())
.limit((max - min).into())
.load::<Post>(conn)
.map_err(Error::from)
}

/// Give a page of posts from a specific instance
pub fn get_instance_page(
conn: &Connection,
instance_id: i32,
(min, max): (i32, i32),
) -> Result<Vec<Post>> {
use schema::blogs;

let blog_ids = blogs::table
.filter(blogs::instance_id.eq(instance_id))
.select(blogs::id);

posts::table
.order(posts::creation_date.desc())
.filter(posts::published.eq(true))
.filter(posts::blog_id.eq_any(blog_ids))
.offset(min.into())
.limit((max - min).into())
.load::<Post>(conn)
.map_err(Error::from)
}

/// Give a page of customized user feed, based on a list of followed users
pub fn user_feed_page(
conn: &Connection,
followed: Vec<i32>,
(min, max): (i32, i32),
) -> Result<Vec<Post>> {
use schema::post_authors;
let post_ids = post_authors::table
.filter(post_authors::author_id.eq_any(followed))
.select(post_authors::post_id);

posts::table
.order(posts::creation_date.desc())
.filter(posts::published.eq(true))
.filter(posts::id.eq_any(post_ids))
.offset(min.into())
.limit((max - min).into())
.load::<Post>(conn)
.map_err(Error::from)
}

pub fn drafts_by_author(conn: &Connection, author: &User) -> Result<Vec<Post>> {
use schema::post_authors;

Expand Down Expand Up @@ -714,6 +652,9 @@ impl FromId<PlumeRocket> for Post {
.ok();
}
}

Timeline::add_to_all_timelines(c, &post, Kind::Original)?;

Ok(post)
}
}
Expand Down
3 changes: 3 additions & 0 deletions plume-models/src/reshares.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ use plume_common::activity_pub::{
};
use posts::Post;
use schema::reshares;
use timeline::*;
use users::User;
use {Connection, Error, PlumeRocket, Result};

Expand Down Expand Up @@ -124,6 +125,8 @@ impl AsObject<User, Announce, &PlumeRocket> for Post {
},
)?;
reshare.notify(conn)?;

Timeline::add_to_all_timelines(c, &self, Kind::Original)?;
trinity-1686a marked this conversation as resolved.
Show resolved Hide resolved
Ok(reshare)
}
}
Expand Down
30 changes: 28 additions & 2 deletions plume-models/src/timeline/mod.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
use diesel::{self, ExpressionMethods, QueryDsl, RunQueryDsl};
use diesel::{self, BoolExpressionMethods, ExpressionMethods, QueryDsl, RunQueryDsl};

use lists::List;
use posts::Post;
Expand All @@ -8,7 +8,9 @@ use {Connection, Error, PlumeRocket, Result};

pub(crate) mod query;

use self::query::{Kind, QueryError, TimelineQuery};
use self::query::{QueryError, TimelineQuery};

pub use self::query::Kind;

#[derive(Clone, Debug, PartialEq, Queryable, Identifiable, AsChangeset)]
#[table_name = "timeline_definition"]
Expand Down Expand Up @@ -68,6 +70,21 @@ impl Timeline {
}
}

/// Same as `list_for_user`, but also includes instance timelines if `user_id` is `Some`.
pub fn list_all_for_user(conn: &Connection, user_id: Option<i32>) -> Result<Vec<Self>> {
if let Some(user_id) = user_id {
timeline_definition::table
.filter(timeline_definition::user_id.eq(user_id).or(timeline_definition::user_id.is_null()))
.load::<Self>(conn)
.map_err(Error::from)
} else {
timeline_definition::table
.filter(timeline_definition::user_id.is_null())
.load::<Self>(conn)
.map_err(Error::from)
}
}

pub fn new_for_user(
conn: &Connection,
user_id: i32,
Expand Down Expand Up @@ -173,6 +190,15 @@ impl Timeline {
.map_err(Error::from)
}

pub fn count_posts(&self, conn: &Connection) -> Result<i64> {
timeline::table
.filter(timeline::timeline_id.eq(self.id))
.inner_join(posts::table)
.count()
.get_result(conn)
.map_err(Error::from)
}

pub fn add_to_all_timelines(rocket: &PlumeRocket, post: &Post, kind: Kind) -> Result<()> {
let timelines = timeline_definition::table
.load::<Self>(rocket.conn.deref())
Expand Down
10 changes: 8 additions & 2 deletions plume-models/src/users.rs
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ use posts::Post;
use safe_string::SafeString;
use schema::users;
use search::Searcher;
use timeline::Timeline;
use {ap_url, Connection, Error, PlumeRocket, Result};

pub type CustomPerson = CustomObject<ApSignature, Person>;
Expand Down Expand Up @@ -927,7 +928,7 @@ impl NewUser {
password: String,
) -> Result<User> {
let (pub_key, priv_key) = gen_keypair();
User::insert(
let res = User::insert(
conn,
NewUser {
username,
Expand All @@ -943,7 +944,12 @@ impl NewUser {
private_key: Some(String::from_utf8(priv_key).or(Err(Error::Signature))?),
..NewUser::default()
},
)
)?;

// create default timeline
Timeline::new_for_user(conn, res.id, "My feed".into(), "followed".into())?;

Ok(res)
}
}

Expand Down
60 changes: 35 additions & 25 deletions po/plume/ar.po
Original file line number Diff line number Diff line change
Expand Up @@ -37,10 +37,27 @@ msgstr "أشار إليك {0}."
msgid "{0} boosted your article."
msgstr ""

msgid "Your feed"
msgstr "خيطك"

msgid "Local feed"
msgstr "الخيط المحلي"

msgid "Federated feed"
msgstr "الخيط الموحد"

# src/template_utils.rs:108
msgid "{0}'s avatar"
msgstr "الصورة الرمزية لـ {0}"

# src/template_utils.rs:195
msgid "Previous page"
msgstr ""

# src/template_utils.rs:206
msgid "Next page"
msgstr ""

# src/routes/blogs.rs:70
msgid "To create a new blog, you need to be logged in"
msgstr "لإنشاء مدونة جديدة، تحتاج إلى تسجيل الدخول"
Expand Down Expand Up @@ -302,14 +319,8 @@ msgstr "مرحبا بكم في {0}"
msgid "Latest articles"
msgstr "آخر المقالات"

msgid "Your feed"
msgstr "خيطك"

msgid "Federated feed"
msgstr "الخيط الموحد"

msgid "Local feed"
msgstr "الخيط المحلي"
msgid "View all"
msgstr "عرضها كافة"

msgid "Administration of {0}"
msgstr "إدارة {0}"
Expand All @@ -332,15 +343,6 @@ msgstr "حظر"
msgid "Ban"
msgstr "اطرد"

msgid "All the articles of the Fediverse"
msgstr "كافة مقالات الفديفرس"

msgid "Articles from {}"
msgstr "مقالات صادرة مِن {}"

msgid "Nothing to see here yet. Try subscribing to more people."
msgstr ""

# src/template_utils.rs:217
msgid "Name"
msgstr "الاسم"
Expand Down Expand Up @@ -568,9 +570,6 @@ msgstr "لا شيء"
msgid "No description"
msgstr "مِن دون وصف"

msgid "View all"
msgstr "عرضها كافة"

msgid "By {0}"
msgstr "مِن طرف {0}"

Expand Down Expand Up @@ -937,12 +936,9 @@ msgstr[5] ""
msgid "No posts to see here yet."
msgstr "في الوقت الراهن لا توجد أية منشورات هنا."

msgid "Query"
msgstr ""

#, fuzzy
msgid "Articles in this timeline"
msgstr "رخصة المقال"
msgid "Nothing to see here yet."
msgstr "في الوقت الراهن لا توجد أية منشورات هنا."

msgid "Articles tagged \"{0}\""
msgstr "المقالات الموسومة بـ \"{0}\""
Expand Down Expand Up @@ -1007,6 +1003,20 @@ msgstr ""
msgid "Use as an avatar"
msgstr "استخدمها كصورة رمزية"

#, fuzzy
#~ msgid "My feed"
#~ msgstr "خيطك"

#~ msgid "All the articles of the Fediverse"
#~ msgstr "كافة مقالات الفديفرس"

#~ msgid "Articles from {}"
#~ msgstr "مقالات صادرة مِن {}"

#, fuzzy
#~ msgid "Articles in this timeline"
#~ msgstr "رخصة المقال"

# src/routes/session.rs:263
#~ msgid "Sorry, but the link expired. Try again"
#~ msgstr "عذراً، ولكن انتهت مدة صلاحية الرابط. حاول مرة أخرى"
Expand Down
Loading