5128bf9d4a
Refactor streaming stuff
...
Now we call the processor once per commit, and it's also now
a commit processor, not an operation processor, so that we can
update the cursor properly
2023-09-22 12:37:10 +02:00
08dc55b2cd
Rejiggle things a bit to make it possible to have multiple binaries here for publishing
2023-09-21 15:01:43 +02:00
901c4b6e97
Make Algo.should_index_post a fallible async function, for maximum extensibility
...
We may want to perform some more complicated operations here in the future
2023-09-21 13:31:27 +02:00
c02bded6f8
Formatting
2023-09-21 13:28:22 +02:00
2fd1474647
Don't crash when unable to classify a profile due to some random problem
...
Random problems include: deleted profiles.
Also always wait 10 seconds between runs, we don't need to do it so often
2023-09-21 13:25:36 +02:00
93c4979c71
Keep subscription state in order to not lose messages
...
This isn't a good way to do it though, because opreations processor is only called for each operation,
so we end up not updating the cursor as often as we realistically should be.
I'll refactor this slightly later
2023-09-21 12:33:17 +02:00
62b00ceed7
Replace random print statements with proper logging setup
2023-09-21 11:22:18 +02:00
f4ee482ce7
Use Arcs to pass stuff around to avoid dealing with lifetimes
...
And also implement proper language detection through lingua-rs,
because Bluesky's detection is really bad
2023-09-21 10:36:47 +02:00
9a2a88dc6b
Mark serving feed as done 🎉
2023-09-16 21:08:32 +02:00
14b9f846da
Refactor things
...
Make the overall design a little more flexible if one wants
to easily implement multiple feeds, and also kinda closer to
the original template provided by Bluesky devs
2023-09-16 21:07:00 +02:00
fb17aa3e6a
Formatting
2023-09-16 17:40:07 +02:00
a11c5446d5
Upgrade dependencies
2023-09-16 17:39:09 +02:00
944be0cd5a
Convert into more modern Rust module structure
...
And also use re-exports liberally for shorter imports
2023-09-16 17:21:18 +02:00
227118dfb1
Rename PostSaver into PostIndexer, which is a better name
2023-09-16 17:16:01 +02:00
b4250e12cd
Serve the feed, according to all the Atproto endpoints
2023-09-16 17:13:57 +02:00
c2899951f6
Read chatgpt and postgresql credentials from .env
2023-09-07 19:20:38 +02:00
832bdf6e92
Simplify lifetimes
...
I think I may have overdone it with lifetime-per-dependency, I doubt that'll ever become necessary
2023-09-07 19:06:39 +02:00
9d3cd75db9
Refactor things a little
...
Split into proper domain components, and 3rd party service glue
2023-09-07 19:02:36 +02:00
3180db4779
Upgrade dependencies, part 2
2023-09-07 19:02:12 +02:00
ba3fbf524d
Upgrade dependencies
2023-09-06 08:30:21 +02:00
46974a9d8c
Detect profiles' likely country of residence through chatgpt
...
In a separate thread
2023-09-06 08:26:03 +02:00
262cd707b4
Rejiggle the imports a bit
2023-08-31 15:16:28 +02:00
a18d448c38
Store profiles in the database as we see them
...
No processing yet, but it's coming soon I guess
2023-08-31 15:11:14 +02:00
6365007fd8
Upgrade dependencies
2023-08-31 15:09:42 +02:00
6394321599
Formatting
2023-08-31 09:56:14 +02:00
e68fa95b97
Update the README a little bit
2023-08-31 09:45:48 +02:00
e5b3db1470
Refactor streaming and start inserting Russian posts into db
2023-08-31 09:42:56 +02:00
1b80bf6ab5
Add a little readme explaining what this project does and what the status is
2023-08-26 18:15:18 +02:00
7f403657aa
Don't crash when cid isn't there
...
Although I don't know why that would happen
2023-08-26 18:01:24 +02:00
a7e5384dfb
Refactor things a tiny little bit
2023-08-19 20:20:27 +02:00
13cef8786c
Initial version of things: just reads the stream of messages and prints it out and that's that
2023-08-18 21:11:49 +02:00
Aleksei Voronov