ADd support for tags

utxo/docs/65cdd331-aa64-48bf-9224-0e1e13c68e41.md

@@ -1,21 +1,26 @@
 # Verax
 
-## Theory
+## Data model
 
-Verax's data model is heavily inspired by Bitcoin's model. It is a data model where transactions are primarily by a set of `payments` that are going to be spend, and a set of new `payments` to be created.
+Verax's data model is heavily inspired by Bitcoin's unspent transaction output
+(UTXO) model. It is a data where transactions are a set of `payments` that are
+spent, and a set of new `payments` to be created.
 
 Each `payment` is a data structure with this information
 
 ```
 struct Payment {
-  created_by_transaction: TransactionId,
-	amount: Amount,
-	destination: Address,
-	spend_by: Option<TransactionId>
+    created_by_transaction: TransactionId,
+    amount: Amount,
+    destination: Address,
+    spend_by: Option<TransactionId>
 }
 ```
 
-Any `payment` that has `spend_by: None` is part of the active balance of each address. Any other payment is not longer part of the active balance, but part of the historial record. These spent payments are read-only from that point forward.
+Any `payment` that has `spend_by: None` is part of the active balance of each
+address. Any other payment is not longer part of the active balance, but part of
+the historial record. These spent payments are read-only from that point
+forward.
 
 Every `payment` is created from a transaction, no exception.
 
@@ -27,11 +32,15 @@ flowchart LR
     B --> |1000 USD| UserB
 ```
 
-After this simple transction, `UserA` cannot longer spend their `1000 USD` and balance, and `UserB` can spend `1000 USD` more.
+After this simple transction, `UserA` cannot longer spend their `1000 USD` and
+balance, and `UserB` can spend `1000 USD` more.
 
-This data model does not care how many payments are being spend or created, as long as the amounts are the same on both ends.
+This data model does not care how many payments are being spend or created, as
+long as the amounts are the same on both ends.
 
-In the following example UserA transfer `1000 USD` to `UserB`, but `1 USD` is deducted from the transfer by the system and that is being transfer to the `FeeManager` account.
+In the following example UserA transfer `1000 USD` to `UserB`, but `1 USD` is
+deducted from the transfer by the system and that is being transfer to the
+`FeeManager` account.
 
 ```mermaid
 flowchart LR
@@ -42,7 +51,10 @@ flowchart LR
 
 ### Multi-step transactions
 
-As mentioned before, the transaction can spend multiple payments and can create multiple as well. As long as the amounts are equal on both ends (in this case `1000 USD, 980 EUR` is equal to `999USD + 1 USD, 979EUR + 1EUR`), the transaction will be valid.
+As mentioned before, the transaction can spend multiple payments and can create
+multiple as well. As long as the amounts are equal on both ends (in this case
+`1000 USD, 980 EUR` is equal to `999USD + 1 USD, 979EUR + 1EUR`), the
+transaction will be valid.
 
 ```mermaid
 flowchart LR
@@ -54,20 +66,28 @@ flowchart LR
     B --> |1 EUR| FeeManager
 ```
 
-When the transaction will be attempted to be persisted, the storage layer will make sure to flag `UserA'` and `UserB'` payments. If that operation fails, the whole transaction creation fails.
+When the transaction will be attempted to be persisted, the storage layer will
+make sure to flag `UserA'` and `UserB'` payments. If that operation fails, the
+whole transaction creation fails.
 
 ## Concurrency model
 
-Because Verax is heavily inspired in Bitcoin's model, the concurrency model is quite simple. When a new transaction is commited into the database, each payment in `input` section is attempted to be spent (altering their `spend_by` field from `None` to `Some(new_transaction_id)`). If any `payment is already spent, or not valid, the whole transaction creation fails and a rollback is issued upper stream. The storage layer ensures that transaction creation and updates are atomic and updates.
+Because Verax is heavily inspired in Bitcoin's model, the concurrency model is
+quite simple. When a new transaction is commited into the database, each payment
+in `input` section is attempted to be spent (altering their `spend_by` field
+from `None` to `Some(new_transaction_id)`). If any `payment is already spent, or
+not valid, the whole transaction creation fails and a rollback is issued upper
+stream. The storage layer ensures that transaction creation and updates are
+atomic and updates.
 
 ```mermaid
 sequenceDiagram
-    Transaction->>+ DB: 
+    Transaction ->>+ DB:s
     critical Spend each input
         loop Spend Inputs
             Transaction ->>+ DB: Spend input
             DB ->>+ Transaction: OK
-        end 
+        end
         DB ->>+  Transaction: OK
         loop Output
             Transaction ->>+ DB: Creates new output
@@ -79,13 +99,23 @@ sequenceDiagram
         DB ->>- Transaction: Error
         Transaction ->>+ DB: Rollback
     end
-    
- 
+
+
 ```
 
-Because of the `input` and `output` model, there is no need to check if the account has enough balance, and there is no need to enforce any locking mechanism, as long as each selected `payment` can be spendable at transaction storing time, the transaction will be created atomically. Each payment in the `inputs` must spendable, or else the whole operation fails, because every update is atomic, ensured by the storage layer.
+Because of the `input` and `output` model, there is no need to check if the
+account has enough balance, and there is no need to enforce any locking
+mechanism, as long as each selected `payment` can be spendable at transaction
+storing time, the transaction will be created atomically. Each payment in the
+`inputs` must spendable, or else the whole operation fails, because every update
+is atomic, ensured by the storage layer.
 
 The conditions for a payment ot be spendable are:
 
-* It must be unspent: Payments are spendable once.
-* It must be finalized: This means that the transaction which created this new `payment` is settled. Any other state is not acceptable and will render this payment not usable.
+- It must be unspent: Payments are spendable once.
+- It must be finalized: This means that the transaction which created this new
+  `payment` is settled. Any other state is not acceptable and will render this
+
+No global state knowledge is required to be sure that no asset is being created
+or destroyed by mistake, as long as the inputs are spendable and the sum of
+amounts inside inputs and outputs matches.

utxo/src/ledger.rs

@@ -264,7 +264,7 @@ where
         };
         let r = self
             .storage
-            .get_transactions(account_id, types)
+            .get_transactions(account_id, &types, &[])
             .await?
             .into_iter()
             .map(|x| x.try_into())

utxo/src/sqlite/batch.rs

@@ -216,6 +216,49 @@ impl<'a> storage::Batch<'a> for Batch<'a> {
         Ok(())
     }
 
+    async fn tag_transaction(
+        &mut self,
+        transaction: &Transaction,
+        tags: &[String],
+    ) -> Result<(), Error> {
+        for tag in tags.iter() {
+            sqlx::query(
+                r#"
+        INSERT INTO "transaction_tags"("transaction_id", "tag", "status", "type")
+        VALUES(?, ?, ?, ?)
+        ON CONFLICT("transaction_id", "tag")
+            DO NOTHING
+        "#,
+            )
+            .bind(transaction.id().to_string())
+            .bind(tag)
+            .bind::<u32>(transaction.status().into())
+            .bind::<u32>(transaction.typ().into())
+            .execute(&mut *self.inner)
+            .await
+            .map_err(|e| Error::Storage(e.to_string()))?;
+        }
+
+        let delete_tags = if tags.is_empty() {
+            r#"DELETE FROM "transaction_tags" WHERE "transaction_id" = ?"#.to_owned()
+        } else {
+            format!(
+                r#"DELETE FROM "transaction_tags" WHERE "transaction_id" = ? AND tag NOT IN (?{})"#,
+                ", ?".repeat(tags.len() - 1)
+            )
+        };
+
+        let mut sql = sqlx::query(&delete_tags).bind(transaction.id().to_string());
+        for tag in tags.iter() {
+            sql = sql.bind(tag);
+        }
+
+        sql.execute(&mut *self.inner)
+            .await
+            .map_err(|e| Error::Storage(e.to_string()))?;
+        Ok(())
+    }
+
     async fn relate_account_to_transaction(
         &mut self,
         transaction: &Transaction,

utxo/src/sqlite/mod.rs

@@ -90,6 +90,16 @@ impl<'a> SQLite<'a> {
                 PRIMARY KEY ("id")
             );
             CREATE INDEX IF NOT EXISTS "changelog_object_id" ON "changelog" ("object_id", "created_at");
+            CREATE TABLE IF NOT EXISTS "transaction_tags" (
+                "transaction_id" VARCHAR(66) NOT NULL,
+                "tag" TEXT NOT NULL,
+                "type" INTEGER NOT NULL,
+                "status" INTEGER NOT NULL,
+                "created_at" DATETIME DEFAULT CURRENT_TIMESTAMP,
+                "updated_at" DATETIME DEFAULT CURRENT_TIMESTAMP,
+                PRIMARY KEY ("transaction_id", "tag")
+            );
+            CREATE INDEX IF NOT EXISTS "tags_index" ON "transaction_tags" ("tag", "type", "status");
         "#,
         )
         .await
@@ -99,6 +109,25 @@ impl<'a> SQLite<'a> {
     }
 
     #[inline]
+    async fn get_tags(
+        &self,
+        conn: &mut SqliteConnection,
+        transaction: &TransactionId,
+    ) -> Result<Vec<String>, Error> {
+        Ok(
+            sqlx::query(r#"SELECT tag FROM "transaction_tags" WHERE "transaction_id"=?"#)
+                .bind(transaction.to_string())
+                .fetch_all(&mut *conn)
+                .await
+                .map_err(|e| Error::Storage(e.to_string()))?
+                .into_iter()
+                .map(|x| x.try_get::<String, usize>(0))
+                .collect::<Result<Vec<_>, _>>()
+                .map_err(|e| Error::Storage(e.to_string()))?,
+        )
+    }
+
+    #[inline]
     async fn get_changelogs_internal<T: DeserializeOwned + Serialize + Send + Sync>(
         &self,
         conn: &mut SqliteConnection,
@@ -506,6 +535,7 @@ impl<'a> Storage<'a, Batch<'a>> for SQLite<'a> {
             spend,
             create,
             typ,
+            tags: self.get_tags(&mut *conn, transaction_id).await?,
             last_change,
             changelog,
             status,
@@ -518,7 +548,8 @@ impl<'a> Storage<'a, Batch<'a>> for SQLite<'a> {
     async fn get_transactions(
         &self,
         account: &AccountId,
-        types: Vec<Type>,
+        types: &[Type],
+        _tags: &[String],
     ) -> Result<Vec<from_db::Transaction>, Error> {
         let mut conn = self
             .db

utxo/src/storage.rs

@@ -88,6 +88,14 @@ pub trait Batch<'a> {
     /// Stores a transaction
     async fn store_transaction(&mut self, transaction: &Transaction) -> Result<(), Error>;
 
+    /// Sets the tags for a given transaction. Any tag not included in this
+    /// vector should be removed
+    async fn tag_transaction(
+        &mut self,
+        transaction: &Transaction,
+        tags: &[String],
+    ) -> Result<(), Error>;
+
     /// Creates a relationship between an account and a transaction.
     ///
     /// This is a chance to build an index which relates accounts to
@@ -161,7 +169,8 @@ where
     async fn get_transactions(
         &self,
         account: &AccountId,
-        types: Vec<Type>,
+        types: &[Type],
+        tags: &[String],
     ) -> Result<Vec<from_db::Transaction>, Error>;
 
     /// Returns all changelogs associated with a given object_id. The result should be sorted from oldest to newest.
@@ -467,7 +476,7 @@ pub mod test {
         assert_eq!(
             1,
             storage
-                .get_transactions(&account1, vec![Type::Deposit])
+                .get_transactions(&account1, &[Type::Deposit], &[])
                 .await
                 .expect("valid tx")
                 .len()
@@ -475,7 +484,7 @@ pub mod test {
         assert_eq!(
             1,
             storage
-                .get_transactions(&account2, vec![Type::Deposit])
+                .get_transactions(&account2, &[Type::Deposit], &[])
                 .await
                 .expect("valid tx")
                 .len()
@@ -483,7 +492,7 @@ pub mod test {
         assert_eq!(
             0,
             storage
-                .get_transactions(&account3, vec![Type::Deposit])
+                .get_transactions(&account3, &[Type::Deposit], &[])
                 .await
                 .expect("valid tx")
                 .len()
@@ -492,7 +501,7 @@ pub mod test {
             assert_eq!(
                 0,
                 storage
-                    .get_transactions(&account, vec![Type::Withdrawal])
+                    .get_transactions(&account, &[Type::Withdrawal], &[])
                     .await
                     .expect("valid tx")
                     .len()

utxo/src/transaction/from_db.rs

@@ -10,6 +10,7 @@ pub struct Transaction {
     pub typ: Type,
     pub status: Status,
     pub last_change: Vec<u8>,
+    pub tags: Vec<String>,
     pub changelog: Vec<Changelog<ChangelogEntry>>,
     pub created_at: DateTime<Utc>,
     pub updated_at: DateTime<Utc>,

utxo/src/transaction/inner.rs

@@ -51,6 +51,8 @@ pub struct Transaction {
     creates: Vec<Payment>,
     changelog: Vec<Changelog<ChangelogEntry>>,
     status: Status,
+    #[serde(skip_serializing_if = "Vec::is_empty")]
+    tags: Vec<String>,
     #[serde(with = "ts_milliseconds")]
     created_at: DateTime<Utc>,
     #[serde(with = "ts_milliseconds")]
@@ -121,6 +123,7 @@ impl Transaction {
             creates: create,
             reference,
             typ: Type::Deposit,
+            tags: Vec::new(),
             status,
             changelog,
             created_at,
@@ -128,6 +131,11 @@ impl Transaction {
         })
     }
 
+    /// Returns a mutable reference to the tags associated with this transaction
+    pub fn get_tags_mut(&mut self) -> &mut [String] {
+        &mut self.tags
+    }
+
     /// Creates a new external withdrawal transaction
     ///
     /// Burns assets to reflect external withdrawals
@@ -169,6 +177,7 @@ impl Transaction {
             spends: spend,
             creates: vec![],
             typ: Type::Withdrawal,
+            tags: Vec::new(),
             reference,
             status,
             changelog,
@@ -248,6 +257,7 @@ impl Transaction {
             spends: spend,
             typ,
             creates: create,
+            tags: Vec::new(),
             status,
             changelog,
             created_at,
@@ -504,6 +514,7 @@ impl Transaction {
             batch.store_changelogs(&input.changelog).await?;
         }
 
+        batch.tag_transaction(&self, &self.tags).await?;
         batch.store_changelogs(&self.changelog).await?;
         batch.commit().await?;
         Ok(())
@@ -520,6 +531,7 @@ impl TryFrom<from_db::Transaction> for Transaction {
             spends: value.spend,
             creates: value.create,
             reference: value.reference,
+            tags: value.tags,
             status: value.status,
             changelog: sort_changes(value.changelog, value.last_change)?,
             created_at: value.created_at,