group events

This commit is contained in:
Bastian Wagner 2025-12-19 13:14:04 +01:00
parent 5a08e2319f
commit e05ab13d0d
16 changed files with 442 additions and 101 deletions

View File

@ -1,98 +1,194 @@
<p align="center">
<a href="http://nestjs.com/" target="blank"><img src="https://nestjs.com/img/logo-small.svg" width="120" alt="Nest Logo" /></a>
</p>
# Costly Architektur & Entscheidungsgrundlagen
[circleci-image]: https://img.shields.io/circleci/build/github/nestjs/nest/master?token=abc123def456
[circleci-url]: https://circleci.com/gh/nestjs/nest
## Ziel des Projekts
Costly ist eine **Offline-fähige PWA zur Aufteilung von Gruppenausgaben**.
<p align="center">A progressive <a href="http://nodejs.org" target="_blank">Node.js</a> framework for building efficient and scalable server-side applications.</p>
<p align="center">
<a href="https://www.npmjs.com/~nestjscore" target="_blank"><img src="https://img.shields.io/npm/v/@nestjs/core.svg" alt="NPM Version" /></a>
<a href="https://www.npmjs.com/~nestjscore" target="_blank"><img src="https://img.shields.io/npm/l/@nestjs/core.svg" alt="Package License" /></a>
<a href="https://www.npmjs.com/~nestjscore" target="_blank"><img src="https://img.shields.io/npm/dm/@nestjs/common.svg" alt="NPM Downloads" /></a>
<a href="https://circleci.com/gh/nestjs/nest" target="_blank"><img src="https://img.shields.io/circleci/build/github/nestjs/nest/master" alt="CircleCI" /></a>
<a href="https://discord.gg/G7Qnnhy" target="_blank"><img src="https://img.shields.io/badge/discord-online-brightgreen.svg" alt="Discord"/></a>
<a href="https://opencollective.com/nest#backer" target="_blank"><img src="https://opencollective.com/nest/backers/badge.svg" alt="Backers on Open Collective" /></a>
<a href="https://opencollective.com/nest#sponsor" target="_blank"><img src="https://opencollective.com/nest/sponsors/badge.svg" alt="Sponsors on Open Collective" /></a>
<a href="https://paypal.me/kamilmysliwiec" target="_blank"><img src="https://img.shields.io/badge/Donate-PayPal-ff3f59.svg" alt="Donate us"/></a>
<a href="https://opencollective.com/nest#sponsor" target="_blank"><img src="https://img.shields.io/badge/Support%20us-Open%20Collective-41B883.svg" alt="Support us"></a>
<a href="https://twitter.com/nestframework" target="_blank"><img src="https://img.shields.io/twitter/follow/nestframework.svg?style=social&label=Follow" alt="Follow us on Twitter"></a>
</p>
<!--[![Backers on Open Collective](https://opencollective.com/nest/backers/badge.svg)](https://opencollective.com/nest#backer)
[![Sponsors on Open Collective](https://opencollective.com/nest/sponsors/badge.svg)](https://opencollective.com/nest#sponsor)-->
Kernanforderungen:
- Gruppen ohne Accounts (Join per Invite)
- Offline Änderungen möglich
- Späterer Sync ohne Datenverlust
- Erweiterbar für Konflikte, Undo, Mehrgeräte-Nutzung
## Description
---
[Nest](https://github.com/nestjs/nest) framework TypeScript starter repository.
## Architektur Überblick
## Project setup
Costly nutzt eine **Hybrid Event-Driven Architektur**:
```bash
$ npm install
- **Events sind die Quelle der Wahrheit**
- **Relationale Tabellen sind Read-Models**
- Offline-Sync basiert auf **append-only Events**
### Überblick
Client
├─ Offline Outbox (Events)
├─ UI arbeitet mit aktuellem State
└─ Sync: sendet Events + holt neue Events
Server
├─ Event Store (group_events, expense_events, ...)
├─ Read Models (groups, expenses, ...)
└─ Projektion: Events → State
yaml
Code kopieren
---
## Warum Events?
### Nicht: „Wie sieht der Zustand aus?“
### Sondern: „Was ist passiert?“
Beispiel:
- ❌ `Group.name = "Trip"`
- ✅ `GROUP_RENAMED { from: "Urlaub", to: "Trip" }`
### Vorteile
- Offline-Änderungen lassen sich sicher synchronisieren
- Events sind idempotent (keine Doppeländerungen)
- Konflikte sind sichtbar und erklärbar
- Undo / Replay später möglich
- Debugging & Audit inklusive
---
## Event-Log vs. History
- `group_events` **ersetzt** klassische `group_history`
- Keine doppelte Speicherung
- Events *sind* die History
Merksatz:
> **History sagt, was ist.
> Events sagen, was passiert ist.**
---
## Hybrid-Ansatz (bewusst gewählt)
Costly ist **kein dogmatisches Event Sourcing**.
Wir nutzen:
- ✅ Events als Quelle der Wahrheit
- ✅ Materialisierte Tabellen (`groups`, `expenses`) für schnelle Queries
➡️ Best of both worlds.
---
## Aktueller Konflikt-Ansatz
**Last-write-wins**, basierend auf Server-Reihenfolge der Events.
- Client, der später synchronisiert, kann frühere Änderungen überschreiben
- Das ist **bewusst akzeptiert** fürs MVP
Spätere Erweiterung möglich (siehe unten).
---
## Event-Grundstruktur
```ts
GroupEvent {
id: string; // vom Client generiert (UUID/ULID)
groupId: string;
type: string; // z.B. GROUP_CREATED, GROUP_RENAMED
payload: object; // fachliche Daten
actorId?: string; // deviceId / memberId
createdAt: Date; // Server-Zeit
}
```
## Warum Client-generierte IDs?
### Ermöglicht Idempotenz
## Compile and run the project
- Events können gefahrlos erneut gesendet werden
- Wichtig für Offline-Retry
### Warum es weiterhin REST-Endpunkte gibt
Es gibt z. B.:
- PATCH /groups/:id/name
Diese Endpunkte:
- validieren Input
- prüfen Regeln
- erzeugen intern Events
```bash
# development
$ npm run start
Der Client muss Events nicht kennen, solange er online ist.
# watch mode
$ npm run start:dev
➡️ Später können Offline-Events direkt an /events/batch geschickt werden.
# production mode
$ npm run start:prod
## Transaktionen aktueller Stand
Jede fachliche Aktion läuft atomar:
- Event wird gespeichert
- Read Model wird aktualisiert
- Beides in einer DB-Transaktion
Beispiel:
```sql
Code kopieren
renameGroup()
├─ UPDATE groups
└─ INSERT group_events
```
### Erweiterung: Optimistic Concurrency (später)
Problem:
- Zwei Geräte ändern offline denselben Wert
## Run tests
Lösung:
- Client sendet zusätzlich expectedLastEventId
- Server prüft:
- passt → akzeptieren
- passt nicht → 409 Conflict
```bash
# unit tests
$ npm run test
Das ist ohne Architekturwechsel möglich, weil Events genutzt werden.
# e2e tests
$ npm run test:e2e
## Erweiterung: Transaktionen als Event-Gruppe
Wenn später komplexe Änderungen kommen (z. B. mehrere Expenses auf einmal):
# test coverage
$ npm run test:cov
Idee
Mehrere Events gehören logisch zusammen.
```ts
{
transactionId: "tx-123",
type: "EXPENSE_CREATED"
}
{
transactionId: "tx-123",
type: "EXPENSE_SPLIT_UPDATED"
}
```
### Möglichkeiten:
## Deployment
Server behandelt Events mit gleicher transactionId atomar
- Undo / Rollback pro Transaktion möglich
- UI kann „eine Aktion“ anzeigen statt viele Events
When you're ready to deploy your NestJS application to production, there are some key steps you can take to ensure it runs as efficiently as possible. Check out the [deployment documentation](https://docs.nestjs.com/deployment) for more information.
### Erweiterung: Undo / Replay
Da Events append-only sind:
- Undo = Gegen-Event
- Replay = Events neu anwenden → State neu aufbauen
If you are looking for a cloud-based platform to deploy your NestJS application, check out [Mau](https://mau.nestjs.com), our official platform for deploying NestJS applications on AWS. Mau makes deployment straightforward and fast, requiring just a few simple steps:
Beispiel:
- GROUP_RENAMED (A → B)
- Undo → GROUP_RENAMED (B → A)
```bash
$ npm install -g @nestjs/mau
$ mau deploy
```
Wichtige Designregeln
- Repositories kennen keine Events
- Services orchestrieren State + Event
- Events sind append-only
- Read Models dürfen neu aufgebaut werden
- Keine Logik im Controller
With Mau, you can deploy your application in just a few clicks, allowing you to focus on building features rather than managing infrastructure.
Warum das alles?
Weil Costly:
- offline funktionieren soll
- ohne Accounts auskommen soll
- später wachsen können soll
- aber jetzt schon stabil sein muss
## Resources
Diese Architektur ist der kleinste sinnvolle Schritt, um das zu erreichen.
Check out a few resources that may come in handy when working with NestJS:
- Visit the [NestJS Documentation](https://docs.nestjs.com) to learn more about the framework.
- For questions and support, please visit our [Discord channel](https://discord.gg/G7Qnnhy).
- To dive deeper and get more hands-on experience, check out our official video [courses](https://courses.nestjs.com/).
- Deploy your application to AWS with the help of [NestJS Mau](https://mau.nestjs.com) in just a few clicks.
- Visualize your application graph and interact with the NestJS application in real-time using [NestJS Devtools](https://devtools.nestjs.com).
- Need help with your project (part-time to full-time)? Check out our official [enterprise support](https://enterprise.nestjs.com).
- To stay in the loop and get updates, follow us on [X](https://x.com/nestframework) and [LinkedIn](https://linkedin.com/company/nestjs).
- Looking for a job, or have a job to offer? Check out our official [Jobs board](https://jobs.nestjs.com).
## Support
Nest is an MIT-licensed open source project. It can grow thanks to the sponsors and support by the amazing backers. If you'd like to join them, please [read more here](https://docs.nestjs.com/support).
## Stay in touch
- Author - [Kamil Myśliwiec](https://twitter.com/kammysliwiec)
- Website - [https://nestjs.com](https://nestjs.com/)
- Twitter - [@nestframework](https://twitter.com/nestframework)
## License
Nest is [MIT licensed](https://github.com/nestjs/nest/blob/master/LICENSE).
## Status
- [x]Groups als Events
- [x] Rename als Event
- [ ] Expenses als Events
- [ ] Client-Outbox (IndexedDB)
- [ ] Sync Pull (GET /events?since=...)
- [ ] Conflict Handling (optional)

View File

@ -19,7 +19,8 @@
"mysql2": "^3.16.0",
"reflect-metadata": "^0.2.2",
"rxjs": "^7.8.1",
"typeorm": "^0.3.28"
"typeorm": "^0.3.28",
"ulid": "^3.0.2"
},
"devDependencies": {
"@eslint/eslintrc": "^3.2.0",
@ -11467,6 +11468,15 @@
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/ulid": {
"version": "3.0.2",
"resolved": "https://registry.npmjs.org/ulid/-/ulid-3.0.2.tgz",
"integrity": "sha512-yu26mwteFYzBAot7KVMqFGCVpsF6g8wXfJzQUHvu1no3+rRRSFcSV2nKeYvNPLD2J4b08jYBDhHUjeH0ygIl9w==",
"license": "MIT",
"bin": {
"ulid": "dist/cli.js"
}
},
"node_modules/unbzip2-stream": {
"version": "1.4.3",
"resolved": "https://registry.npmjs.org/unbzip2-stream/-/unbzip2-stream-1.4.3.tgz",

View File

@ -30,7 +30,8 @@
"mysql2": "^3.16.0",
"reflect-metadata": "^0.2.2",
"rxjs": "^7.8.1",
"typeorm": "^0.3.28"
"typeorm": "^0.3.28",
"ulid": "^3.0.2"
},
"devDependencies": {
"@eslint/eslintrc": "^3.2.0",

View File

@ -2,8 +2,9 @@ import { Module } from '@nestjs/common';
import { ConfigModule, ConfigService } from '@nestjs/config';
import { TypeOrmModule } from '@nestjs/typeorm';
import { GroupEntity } from '../groups/persistence/group.entity';
import { GroupEventEntity } from 'src/groups/persistence/group-event.entity';
const ENTITY = [GroupEntity]
const ENTITY = [GroupEntity, GroupEventEntity]
@Module({
imports: [
@ -19,7 +20,7 @@ const ENTITY = [GroupEntity]
password: configService.get('DATABASE_PASSWORD'),
database: configService.get('DATABASE_NAME'),
entities: ENTITY,
synchronize: true,
synchronize: (configService.get('DATABASE_SYNC')||'').toLowerCase() == 'true',
})
})
],

View File

@ -1,15 +1,43 @@
import { Body, Controller, Post } from "@nestjs/common";
import { Body, Controller, Get, Param, Patch, Post } from "@nestjs/common";
import { CreateGroupDto } from "../dto/create-group.dto";
import { GroupsService } from "../application/groups.service";
import { RenameGroupDto } from "../dto/rename-group.dto";
import { IncomingEventDto } from "src/model/dto/incoming-event.dto";
import { ulid } from 'ulid';
@Controller({path: 'groups'})
export class GroupsController {
constructor(private groupsService: GroupsService) {}
@Get()
getGroups() {
return this.groupsService.getAll();
}
@Post()
createGroup(@Body() dto: CreateGroupDto) {
return this.groupsService.create(dto);
}
@Patch(':id/name')
rename(@Param('id') id: string, @Body() dto: RenameGroupDto) {
// actorId später z.B. aus Header/Cookie ziehen
const event: IncomingEventDto = {
id: ulid(),
payload: { to: dto.name },
type: 'GROUP_RENAMED',
actorId: undefined
}
return this.ingest(id, [event])
}
@Post(':groupId/events/batch')
ingest(
@Param('groupId') groupId: string,
@Body() events: IncomingEventDto[],
) {
return this.groupsService.ingestEvents(groupId, events);
}
}

View File

@ -1,10 +1,16 @@
import { Injectable, NotFoundException } from "@nestjs/common";
import { GroupsRepository } from "../persistence/group.repository";
import { CreateGroupDto } from "../dto/create-group.dto";
import { UpdateGroupDto } from "../dto/update-group.dto";
import { DataSource } from "typeorm";
import { GroupEntity } from "../persistence/group.entity";
import { GroupEventEntity } from "../persistence/group-event.entity";
import { ulid } from 'ulid';
import { IncomingEventDto } from "src/model/dto/incoming-event.dto";
@Injectable()
export class GroupsService {
constructor(private readonly groups: GroupsRepository) {}
constructor(private readonly dataSource: DataSource, private readonly groups: GroupsRepository) {}
async join(id: string) {
const group = await this.groups.findById(id);
@ -12,7 +18,93 @@ export class GroupsService {
return group;
}
async create(dto: CreateGroupDto) {
return this.groups.create(dto);
async create(dto: CreateGroupDto, actorId?: string) {
return this.dataSource.transaction(async (manager) => {
const created = await this.groups.createGroup(dto, manager);
await manager.getRepository(GroupEventEntity).save({
id: ulid(),
groupId: created.id,
type: 'GROUP_CREATED',
actorId,
payload: {
name: created.name,
},
});
return created;
});
}
async getAll() {
return this.groups.all();
}
// async rename(groupId: string, newName: string, actorId?: string) {
// return this.dataSource.transaction(async (manager) => {
// const group = await this.groups.findByIdOrFail(groupId, manager);
// const from = group.name;
// const to = newName;
// // optional: wenn gleich, nix tun (oder trotzdem eventen, wie du willst)
// if (from === to) return group;
// group.name = to;
// const saved = await this.groups.save(group, manager);
// await manager.getRepository(GroupEventEntity).save({
// id: ulid(),
// groupId: saved.id,
// type: 'GROUP_RENAMED',
// actorId,
// payload: { from, to },
// });
// return saved;
// });
// }
async ingestEvents(groupId: string, events: IncomingEventDto[]) {
return this.dataSource.transaction(async (manager) => {
const eventsRepo = manager.getRepository(GroupEventEntity);
const groupsRepo = manager.getRepository(GroupEntity);
// 1) Events persistieren (idempotent)
// -> INSERT IGNORE: wenn id schon da, wird es ignoriert
await manager
.createQueryBuilder()
.insert()
.into(GroupEventEntity)
.values(events.map(e => ({
id: e.id,
groupId,
type: e.type as any,
payload: e.payload,
actorId: e.actorId,
})))
.orIgnore() // MySQL: INSERT IGNORE
.execute();
// 2) Jetzt nur die neu eingefügten Events anwenden
// Trick: hole Events anhand IDs aus DB und wende in stabiler Reihenfolge an
const persisted = await eventsRepo.find({
where: { id: events.map(e => e.id) as any },
order: { createdAt: 'ASC' },
});
// optional: nur diejenigen anwenden, die "gerade neu" sind
// (Wenn du ganz sauber sein willst: execute() liefert affected, aber nicht IDs.
// Pragmatik: apply ist idempotent, dann ist doppelt anwenden ok, aber nur wenn Operationen idempotent sind!)
for (const ev of persisted) {
if (ev.type === 'GROUP_RENAMED') {
const group = await groupsRepo.findOneByOrFail({ id: groupId });
group.name = ev.payload.to;
await groupsRepo.save(group);
}
}
return { accepted: events.length };
});
}
}

View File

@ -0,0 +1,8 @@
import { IsString, IsNotEmpty, Length } from 'class-validator';
export class RenameGroupDto {
@IsString()
@IsNotEmpty()
@Length(1, 100)
name: string;
}

View File

@ -0,0 +1,6 @@
import { IsOptional, IsString, Length } from "class-validator";
export class UpdateGroupDto {
@IsOptional() @IsString() @Length(1, 100)
name?: string;
}

View File

@ -4,9 +4,10 @@ import { GroupEntity } from './persistence/group.entity';
import { GroupsController } from './api/groups.controller';
import { GroupsService } from './application/groups.service';
import { GroupsRepository } from './persistence/group.repository';
import { GroupEventEntity } from './persistence/group-event.entity';
@Module({
imports: [TypeOrmModule.forFeature([GroupEntity])],
imports: [TypeOrmModule.forFeature([GroupEntity, GroupEventEntity])],
controllers: [GroupsController],
providers: [GroupsService, GroupsRepository]
})

View File

@ -0,0 +1,29 @@
import { Entity, PrimaryColumn, Column, CreateDateColumn, Index } from 'typeorm';
export type GroupEventType =
| 'GROUP_CREATED'
| 'GROUP_RENAMED'
| 'GROUP_INVITE_ROTATED'
| 'GROUP_DELETED';
@Entity('group_events')
@Index(['groupId', 'createdAt'])
export class GroupEventEntity {
@PrimaryColumn()
id: string; // ULID oder UUID, kommt vom Server oder Client
@Column()
groupId: string;
@Column({ type: 'varchar', length: 64 })
type: GroupEventType;
@Column({ type: 'json' })
payload: any;
@Column({ nullable: true })
actorId?: string;
@CreateDateColumn()
createdAt: Date;
}

View File

@ -1,4 +1,4 @@
import { Column, Entity, PrimaryGeneratedColumn } from "typeorm";
import { Column, CreateDateColumn, Entity, PrimaryGeneratedColumn, UpdateDateColumn } from "typeorm";
@Entity('groups')
export class GroupEntity {
@ -9,4 +9,10 @@ export class GroupEntity {
@Column({ nullable: false })
name: string;
@CreateDateColumn({ name: 'created_at' })
createdAt: Date;
@UpdateDateColumn({ name: 'updated_at' })
updatedAt: Date;
}

View File

@ -1,8 +1,9 @@
import { Injectable } from "@nestjs/common";
import { InjectRepository } from "@nestjs/typeorm";
import { Repository } from "typeorm";
import { EntityManager, Repository } from "typeorm";
import { GroupEntity } from "./group.entity";
import { CreateGroupDto } from "../dto/create-group.dto";
import { UpdateGroupDto } from "../dto/update-group.dto";
@Injectable()
export class GroupsRepository {
@ -11,15 +12,36 @@ export class GroupsRepository {
private readonly repo: Repository<GroupEntity>,
) {}
save(group: GroupEntity) {
return this.repo.save(group);
}
findById(id: string) {
return this.repo.findOne({ where: { id } });
}
create(dto: CreateGroupDto) {
return this.repo.save(this.repo.create(dto))
all() {
return this.repo.find();
}
createGroup(
dto: CreateGroupDto,
manager?: EntityManager,
): Promise<GroupEntity> {
const r = manager ? manager.getRepository(GroupEntity) : this.repo;
const group = r.create({
name: dto.name,
});
return r.save(group);
}
private getRepo(manager?: EntityManager) {
return manager ? manager.getRepository(GroupEntity) : this.repo;
}
async findByIdOrFail(id: string, manager?: EntityManager) {
return this.getRepo(manager).findOneByOrFail({ id });
}
async save(group: GroupEntity, manager?: EntityManager) {
return this.getRepo(manager).save(group);
}
}

View File

@ -0,0 +1,15 @@
import { IsString, IsObject, IsOptional, IsISO8601 } from "class-validator";
export type GROUPEVENTTYPE = 'GROUP_RENAMED'
export class IncomingEventDto {
@IsString() id: string;
@IsString() type: GROUPEVENTTYPE;
@IsObject() payload: any;
@IsOptional() @IsString()
actorId?: string;
@IsOptional() @IsISO8601()
clientCreatedAt?: string;
}

View File

@ -1,26 +1,26 @@
services:
frontend:
build:
context: ./frontend
image: gitea.bastian-wagner.dev/bastian/valgard/frontend:latest
context: ./costly-api
image: gitea.forgecore.work/bastian/costly/costly-client:latest
networks:
- internal
backend:
build:
context: ./backend
image: gitea.bastian-wagner.dev/bastian/valgard/backend:latest
image: gitea.forgecore.work/bastian/costly/costly-api:latest
networks:
- internal
nginx:
image: gitea.bastian-wagner.dev/bastian/valgard/nginx:latest
image: gitea.forgecore.work/bastian/costly/nginx:latest
build:
context: ./nginx
ports:
- "8081:80"
- "8082:80"
volumes:
- ./nginx/default.conf:/etc/nginx/conf.d/default.conf
- ./nginx.conf:/etc/nginx/conf.d/default.conf
depends_on:
- frontend
- backend

21
package-lock.json generated Normal file
View File

@ -0,0 +1,21 @@
{
"name": "costly",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"dependencies": {
"ulid": "^3.0.2"
}
},
"node_modules/ulid": {
"version": "3.0.2",
"resolved": "https://registry.npmjs.org/ulid/-/ulid-3.0.2.tgz",
"integrity": "sha512-yu26mwteFYzBAot7KVMqFGCVpsF6g8wXfJzQUHvu1no3+rRRSFcSV2nKeYvNPLD2J4b08jYBDhHUjeH0ygIl9w==",
"license": "MIT",
"bin": {
"ulid": "dist/cli.js"
}
}
}
}

5
package.json Normal file
View File

@ -0,0 +1,5 @@
{
"dependencies": {
"ulid": "^3.0.2"
}
}