🧶 Library to handling files for persistent storage with Google Cloud Storage and Amazon S3-compatible server, made in Kotlin

Noelware

Last update: Dec 17, 2022

Related tags

Kotlin Projects kotlin storage s3 google-cloud kotlin-library gcs

Overview

🧶 Remi

Library to handling files for persistent storage with Google Cloud Storage and Amazon S3-compatible server, made in Kotlin!

Why is this built?

This was built in the mind of Arisu and helm-server to store any piece of data into different providers without repeating code and so on, and so forth. This is a simple library to take care of that!

We support any Amazon S3 compatible servers, Google Cloud Storage, and the local filesystem!

The library is split into five modules:

remi-bom - Bill of Materials for Remi
remi-core - Core abstractions that is used for the storage units.
remi-support-fs - Support for using the local filesystem
remi-support-gcs - Support for using Google Cloud Storage
remi-support-s3 - Support for using any compatible Amazon S3 service like Wasabi or MinIO

Installation

📜 Documentation | 👀 v0.0.1

Gradle

Kotlin DSL

 ` // in the dependency declaration implementation("org.noelware.remi:remi-
  :
  
   ") }"> 
   repositories {
    // If you're using the Noel Gradle Utils package, you can use the
    // `noelware` extension
    maven {
        url = uri("https://maven.noelware.org")
    }
}

dependencies {
    // If you're using the Noel Gradle Utils package, you can use
    // the `noelware` extension to automatically prefix `org.noelware.
     
      `
     
    // in the dependency declaration
    implementation("org.noelware.remi:remi-
     
      :
      
       "
      
     )
} 
  
 

Groovy DSL

 :
  " }"> 
  repositories {
    maven {
        url "https://maven.noelware.org"
    }
}

dependencies {
    implementation "org.noelware.remi:remi-
    
     :
     
      "
     
    
} 
 

Maven

Declare the Noelware Maven repository under the chain:

<repositories>
    <repository>
        <id>noelware-mavenid>
        <url>https://maven.noelware.orgurl>
    repository>
repositories>

Now declare the dependency you want under the chain:

<dependencies>
    <dependency>
        <groupId>org.noelware.remigroupId>
        <artifactId>remi-{{NAME}}artifactId>
        <version>{{VERSION}}version>
        <type>pomtype>
    dependency>
dependencies>

License

remi is released under the Apache 2.0 License by Noelware, read the LICENSE file in the root repository for more information.

Comments

Retry spam on every executive request to GetObject

This is related to a problem while developing hazel:

I feel like calling GetObject to get the original input stream every time the object is iterated on ListObjectResponse.contents():

            for (content in res.contents()) {
                // TODO: is this slow for >50mb objects?
                // TODO: find another way to get the input stream (for Tika)
                val obj = try {
                    client.getObject({
                        it.bucket(config.bucket)
                        it.key(content.key())
                    }, ResponseTransformer.toInputStream())
                } catch (e: Exception) {
                    null
                } ?: continue

While testing this for the library, I couldn't find an way to get the input stream from S3Object once it was iterated, so what should the library be doing for listAll(): List<org.noelware.remi.core.Object>?

bug good first issue help wanted

opened by auguwu 1

Configure Renovate
Welcome to Renovate! This is an onboarding PR to help you understand and configure settings before regular Pull Requests begin.

🚦 To activate Renovate, merge this Pull Request. To disable Renovate, simply close this Pull Request unmerged.

Detected Package Files

docker-compose.yml (docker-compose)

.github/workflows/ktlint.yml (github-actions)

.github/workflows/release-snapshot.yml (github-actions)

.github/workflows/release-stable.yml (github-actions)

.github/workflows/update-docs.yml (github-actions)

gradle.properties (gradle)

settings.gradle.kts (gradle)

build.gradle.kts (gradle)

bom/build.gradle.kts (gradle)

buildSrc/build.gradle.kts (gradle)

buildSrc/src/main/kotlin/remi-module.gradle.kts (gradle)

buildSrc/src/main/kotlin/remi-publishing.gradle.kts (gradle)

core/build.gradle.kts (gradle)

support-fs/build.gradle.kts (gradle)

support-gcs/build.gradle.kts (gradle)

support-s3/build.gradle.kts (gradle)

gradle/wrapper/gradle-wrapper.properties (gradle-wrapper)

Configuration Summary

Based on the default config's presets, Renovate will:

Start dependency updates only once this onboarding PR is merged

Enable Renovate Dependency Dashboard creation

If semantic commits detected, use semantic commit type fix for dependencies and chore for all others

Ignore node_modules, bower_components, vendor and various test/tests directories

Autodetect whether to pin dependencies or maintain ranges

Rate limit PR creation to a maximum of two per hour

Limit to maximum 10 open PRs at any time

Group known monorepo packages together

Use curated list of recommended non-monorepo package groupings

Fix some problems with very old Maven commons versions

Ignore spring cloud 1.x releases

Ignore web3j 5.0.0 release

Ignore http4s digest-based 1.x milestones

Use node versioning for @types/node

Limit concurrent requests to reduce load on Repology servers until we can fix this properly, see issue 10133

Do not upgrade from Alpine stable to edge

🔡 Would you like to change the way Renovate is upgrading your dependencies? Simply edit the renovate.json in this branch with your custom config and the list of Pull Requests in the "What to Expect" section below will be updated the next time Renovate runs.

What to Expect

With your current configuration, Renovate will create 7 Pull Requests:

chore(deps): update dependency org.jetbrains.kotlinx:kotlinx-coroutines-bom to v1.6.0

Schedule: ["at any time"]

Branch name: renovate/org.jetbrains.kotlinx-kotlinx-coroutines-bom-1.x

Merge into: master

Upgrade org.jetbrains.kotlinx:kotlinx-coroutines-bom to 1.6.0

chore(deps): update dependency org.jetbrains.kotlinx:kotlinx-serialization-bom to v1.3.2

Schedule: ["at any time"]

Branch name: renovate/org.jetbrains.kotlinx-kotlinx-serialization-bom-1.x

Merge into: master

Upgrade org.jetbrains.kotlinx:kotlinx-serialization-bom to 1.3.2

chore(deps): update dependency software.amazon.awssdk:bom to v2.17.143

Schedule: ["at any time"]

Branch name: renovate/aws-java-sdk-v2-monorepo

Merge into: master

Upgrade software.amazon.awssdk:bom to 2.17.143

chore(deps): update dependency gradle to v7.4

Schedule: ["at any time"]

Branch name: renovate/gradle-7.x

Merge into: master

Upgrade gradle to 7.4

chore(deps): update dependency io.kotest:kotest-bom to v5.1.0

Schedule: ["at any time"]

Branch name: renovate/io.kotest-kotest-bom-5.x

Merge into: master

Upgrade io.kotest:kotest-bom to 5.1.0

chore(deps): update actions/checkout action to v3

Schedule: ["at any time"]

Branch name: renovate/actions-checkout-3.x

Merge into: master

Upgrade actions/checkout to v3

chore(deps): update actions/setup-java action to v3

Schedule: ["at any time"]

Branch name: renovate/actions-setup-java-3.x

Merge into: master

Upgrade actions/setup-java to v3

🚸 Branch creation will be limited to maximum 2 per hour, so it doesn't swamp any CI resources or spam the project. See docs for prhourlylimit for details.

❓ Got questions? Check out Renovate's Docs, particularly the Getting Started section. If you need any further assistance then you can also request help here.

This PR has been generated by WhiteSource Renovate. View repository job log here.
opened by renovate[bot] 1
fix(deps): update dependency software.amazon.awssdk:s3 to v2.19.9
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | software.amazon.awssdk:s3 (source) | 2.19.8 -> 2.19.9 | | | | |

Release Notes

aws/aws-sdk-java-v2

v2.19.9

Compare Source

AWS SDK for Java v2

Features

Updated endpoint and partition metadata.

Amazon Security Lake

Features

Allow CreateSubscriber API to take string input that allows setting more descriptive SubscriberDescription field. Make souceTypes field required in model level for UpdateSubscriberRequest as it is required for every API call on the backend. Allow ListSubscribers take any String as nextToken param.

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Enabled.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate. View repository job log here.
opened by renovate[bot] 0
chore(deps): update dependency com.diffplug.spotless:spotless-plugin-gradle to v6.12.1
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | com.diffplug.spotless:spotless-plugin-gradle | 6.12.0 -> 6.12.1 | | | | |

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Enabled.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate. View repository job log here.
opened by renovate[bot] 0
chore(deps): update dependency dev.floofy.commons:gradle to v2.4.2
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | dev.floofy.commons:gradle (source) | 2.4.1 -> 2.4.2 | | | | |

Release Notes

auguwu/common-utils

v2.4.2

Compare Source

:bug: Infrastructure

Upgrade Kotlin to v1.8.0 (@auguwu, @renovate-bot; #61)

Upgrade dependency io.insert-koin:koin-core to v3.3.2 (@renovate-bot; #62)

Full Changelog: https://github.com/auguwu/common-utils/compare/2.4.1...2.4.2

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Enabled.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate. View repository job log here.
opened by renovate[bot] 0
fix(deps): update dependency software.amazon.awssdk:s3 to v2.19.8
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | software.amazon.awssdk:s3 (source) | 2.19.7 -> 2.19.8 | | | | |

Release Notes

aws/aws-sdk-java-v2

v2.19.8

Compare Source

AWS IoT FleetWise

Features

Update documentation - correct the epoch constant value of default value for expiryTime field in CreateCampaign request.

AWS SDK for Java v2

Features

Updated endpoint and partition metadata.

Amazon CloudFront

Features

Extend response headers policy to support removing headers from viewer responses

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Enabled.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate. View repository job log here.
opened by renovate[bot] 0
fix(deps): update dependency software.amazon.awssdk:s3 to v2.19.7
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | software.amazon.awssdk:s3 (source) | 2.19.6 -> 2.19.7 | | | | |

Release Notes

aws/aws-sdk-java-v2

v2.19.7

Compare Source

AWS SDK for Java v2

Features

Updated endpoint and partition metadata.

AWS Secrets Manager

Features

Added owning service filter, include planned deletion flag, and next rotation date response parameter in ListSecrets.

Amazon API Gateway

Features

Documentation updates for Amazon API Gateway

Amazon Connect Wisdom Service

Features

This release extends Wisdom CreateContent and StartContentUpload APIs to support PDF and MicrosoftWord docx document uploading.

Amazon EMR

Features

Added GetClusterSessionCredentials API to allow Amazon SageMaker Studio to connect to EMR on EC2 clusters with runtime roles and AWS Lake Formation-based access control for Apache Spark, Apache Hive, and Presto queries.

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Enabled.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate. View repository job log here.
opened by renovate[bot] 0
fix(deps): update dependency software.amazon.awssdk:s3 to v2.19.6
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | software.amazon.awssdk:s3 (source) | 2.19.5 -> 2.19.6 | | | | |

Release Notes

aws/aws-sdk-java-v2

v2.19.6

Compare Source

AWS Network Firewall

Features

AWS Network Firewall now provides status messages for firewalls to help you troubleshoot when your endpoint fails.

AWS Route53 Recovery Control Config

Features

Added support for Python paginators in the route53-recovery-control-config List* APIs.

Amazon ElastiCache

Features

This release allows you to modify the encryption in transit setting, for existing Redis clusters. You can now change the TLS configuration of your Redis clusters without the need to re-build or re-provision the clusters or impact application availability.

Amazon Relational Database Service

Features

This release adds support for Custom Engine Version (CEV) on RDS Custom SQL Server.

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Enabled.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate. View repository job log here.
opened by renovate[bot] 0
fix(deps): update dependency software.amazon.awssdk:s3 to v2.19.5
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | software.amazon.awssdk:s3 (source) | 2.19.4 -> 2.19.5 | | | | |

Release Notes

aws/aws-sdk-java-v2

v2.19.5

Compare Source

AWS Transfer Family

Features

Add additional operations to throw ThrottlingExceptions

Amazon MemoryDB

Features

This release adds support for MemoryDB Reserved nodes which provides a significant discount compared to on-demand node pricing. Reserved nodes are not physical nodes, but rather a billing discount applied to the use of on-demand nodes in your account.

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Enabled.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate. View repository job log here.
opened by renovate[bot] 0
fix(deps): update dependency software.amazon.awssdk:s3 to v2.19.4
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | software.amazon.awssdk:s3 (source) | 2.19.3 -> 2.19.4 | | | | |

Release Notes

aws/aws-sdk-java-v2

v2.19.4

Compare Source

AWS SDK for Java v2

Features

Updated endpoint and partition metadata.

Amazon Connect Participant Service

Features

Amazon Connect Chat introduces the Message Receipts feature. This feature allows agents and customers to receive message delivered and read receipts after they send a chat message.

Amazon Connect Service

Features

Support for Routing Profile filter, SortCriteria, and grouping by Routing Profiles for GetCurrentMetricData API. Support for RoutingProfiles, UserHierarchyGroups, and Agents as filters, NextStatus and AgentStatusName for GetCurrentUserData. Adds ApproximateTotalCount to both APIs.

Amazon Detective

Features

This release adds a missed AccessDeniedException type to several endpoints.

Amazon FSx

Features

Fix a bug where a recent release might break certain existing SDKs.

Inspector2

Features

Amazon Inspector adds support for scanning NodeJS 18.x and Go 1.x AWS Lambda function runtimes.

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

?? Automerge: Enabled.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate. View repository job log here.
opened by renovate[bot] 0
fix(deps): update dependency software.amazon.awssdk:s3 to v2.19.3
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | software.amazon.awssdk:s3 (source) | 2.19.2 -> 2.19.3 | | | | |

Release Notes

aws/aws-sdk-java-v2

v2.19.3

Compare Source

AWS Compute Optimizer

Features

This release enables AWS Compute Optimizer to analyze and generate optimization recommendations for ecs services running on Fargate.

AWS IoT Core Device Advisor

Features

This release adds the following new features: 1) Documentation updates for IoT Device Advisor APIs. 2) Updated required request parameters for IoT Device Advisor APIs. 3) Added new service feature: ability to provide the test endpoint when customer executing the StartSuiteRun API.

AWS SDK for Java v2

Features

Updated endpoint and partition metadata.

AWS Secrets Manager

Features

Documentation updates for Secrets Manager

Amazon Connect Service

Features

Amazon Connect Chat introduces the Idle Participant/Autodisconnect feature, which allows users to set timeouts relating to the activity of chat participants, using the new UpdateParticipantRoleConfig API.

Amazon Kinesis Video WebRTC Storage

Features

Amazon Kinesis Video Streams offers capabilities to stream video and audio in real-time via WebRTC to the cloud for storage, playback, and analytical processing. Customers can use our enhanced WebRTC SDK and cloud APIs to enable real-time streaming, as well as media ingestion to the cloud.

Amazon Relational Database Service

Features

Add support for managing master user password in AWS Secrets Manager for the DBInstance and DBCluster.

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Enabled.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate. View repository job log here.
opened by renovate[bot] 0
chore(deps): update plugin com.gradle.enterprise to v3.12.2
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | com.gradle.enterprise | 3.12.1 -> 3.12.2 | | | | |

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Enabled.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate. View repository job log here.
opened by renovate[bot] 0
Cached-based storage trailer

This will allow people to have a "basic cache" mechanism for storage since it can consume a lot of I/O, so having a cache for objects will be faster. :)

It should use Caffeine as the cache provider (which it can be customized). This will be under the new :cache-trailer package.
documentation enhancement

opened by auguwu 0
Dependency Dashboard
This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

[ ] chore(deps): update plugin com.gradle.enterprise to v3.12.2

Detected dependencies

github-actions

.github/workflows/Linting.yaml

actions/checkout v3

actions/setup-java v3

gradle/gradle-build-action v2

gradle/wrapper-validation-action v1

gradle/gradle-build-action v2

gradle/gradle-build-action v2

gradle/gradle-build-action v2

.github/workflows/ReleaseNightly.yaml

actions/checkout v3

actions/setup-java v3

gradle/gradle-build-action v2

.github/workflows/ReleaseStable.yaml

actions/checkout v3

actions/setup-java v3

gradle/gradle-build-action v2

gradle

gradle.properties

settings.gradle.kts

com.gradle.enterprise 3.12.1

build.gradle.kts

bom/build.gradle.kts

buildSrc/build.gradle.kts

com.diffplug.spotless:spotless-plugin-gradle 6.12.1

dev.floofy.commons:gradle 2.4.2

buildSrc/src/main/kotlin/remi-module.gradle.kts

org.jetbrains:annotations 23.1.0

org.slf4j:slf4j-api 2.0.6

org.junit.jupiter:junit-jupiter-api 5.9.1

org.junit.jupiter:junit-jupiter-engine 5.9.1

org.slf4j:slf4j-simple 2.0.6

core/build.gradle.kts

org.apache.tika:tika-core 2.6.0

support/azure/build.gradle.kts

org.testcontainers:testcontainers 1.17.6

org.testcontainers:junit-jupiter 1.17.6

com.azure:azure-storage-blob 12.20.1

support/fs/build.gradle.kts

support/gcs/build.gradle.kts

com.google.cloud:google-cloud-storage 2.16.0

support/s3/build.gradle.kts

org.testcontainers:testcontainers 1.17.6

org.testcontainers:junit-jupiter 1.17.6

software.amazon.awssdk:s3 2.19.9

gradle-wrapper

gradle/wrapper/gradle-wrapper.properties

gradle 7.6

[ ] Check this box to trigger a request for Renovate to run again on this repository
opened by renovate[bot] 0

Releases(v0.5.4-beta)

v0.5.4-beta(Dec 17, 2022)
fix the #blob() method in amazon s3 storage to return null if NoSuchKeyException had been thrown

Dependency Updates

Update dependency software.amazon.awssdk:s3 to v2.18.41 (@renovate-bot, #271)

Update dependency org.jetbrains:annotations to v23.1.0 (@renovate-bot, #272)

Full Changelog: https://github.com/Noelware/remi/compare/v0.5.3-beta...v0.5.4-beta
Source code(tar.gz)
Source code(zip)
v0.5.3-beta(Dec 17, 2022)
disabled etag encoding in file system storage service (@auguwu)

Full Changelog: https://github.com/Noelware/remi/compare/v0.5.2-beta...v0.5.3-beta
Source code(tar.gz)
Source code(zip)
v0.5.2-beta(Dec 17, 2022)

This fixes Etag encoding in the filesystem storage service
Source code(tar.gz)
Source code(zip)
v0.5.1-beta(Dec 17, 2022)

This release just updates the license years and other stuff.
Source code(tar.gz)
Source code(zip)
v0.5-snapshot(Dec 12, 2022)

This contains the next release of Remi, a full infrastructure refractor of the project. It is now purely made in Java to let Java users use the power of Remi and not have suspend functions.

The project also contains Kotlin extensions and kotlinx.serialization support for configurations for the supported storage services.

This release also implements Azure and Google Cloud Storage support.
Source code(tar.gz)
Source code(zip)
v0.4.1-beta(Jul 30, 2022)
This release doesn't add much, it only fixes #normalizePath(String) to be more accurate.

Dependency Updates

chore(deps): update dependency com.diffplug.spotless:spotless-plugin-gradle from v6.8.0 to v6.9.0 by @renovate-bot

chore(deps): update dependency io.kotest:kotest-bom to from v5.3.2 to v5.4.1 by @renovate-bot

fix(deps): update dependency software.amazon.awssdk:s3 from v2.17.237 to v2.17.243 by @renovate-bot

Full Changelog: https://github.com/Noelware/remi/compare/v0.4-beta...v0.4.1-beta
Source code(tar.gz)
Source code(zip)
v0.4-beta(Jul 22, 2022)
Additions

feat: add #stats function in filesystem storage trailer by @auguwu

feat: throw error if directory is readonly by @auguwu

What's Changed

chore(deps): update dependency com.google.cloud:google-cloud-storage to v2.9.3 by @renovate-bot in https://github.com/Noelware/remi/pull/135

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.228 by @renovate-bot in https://github.com/Noelware/remi/pull/136

chore(deps): update dependency io.minio:minio to v8.4.3 by @renovate-bot in https://github.com/Noelware/remi/pull/137

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.229 by @renovate-bot in https://github.com/Noelware/remi/pull/138

chore(deps): update dependency gradle to v7.5-rc-5 by @renovate in https://github.com/Noelware/remi/pull/139

chore(deps): update dependency org.jetbrains.dokka:dokka-gradle-plugin to v1.7.10 by @renovate-botin https://github.com/Noelware/remi/pull/140

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.230 by @renovate-bot in https://github.com/Noelware/remi/pull/141

chore(deps): update dependency org.jetbrains.kotlinx:kotlinx-coroutines-bom to v1.6.4 by @renovate-bot in https://github.com/Noelware/remi/pull/142

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.231 by @renovate-bot in https://github.com/Noelware/remi/pull/143

chore(deps): update dependency dev.floofy.commons:gradle to v2.2.1 by @renovate-bot in https://github.com/Noelware/remi/pull/144

chore(deps): update dependency dev.floofy.commons:slf4j to v2.2.1 by @renovate-bot in https://github.com/Noelware/remi/pull/145

chore(deps): update dependency gradle to v7.5 by @renovate-bot in https://github.com/Noelware/remi/pull/146

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.232 by @renovate-bot in https://github.com/Noelware/remi/pull/147

chore(deps): update dependency com.google.cloud:google-cloud-storage to v2.10.0 by @renovate-bot in https://github.com/Noelware/remi/pull/148

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.233 by @renovate-bot in https://github.com/Noelware/remi/pull/149

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.234 by @renovate-bot in https://github.com/Noelware/remi/pull/150

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.235 by @renovate-bot in https://github.com/Noelware/remi/pull/151

chore(deps): update dependency dev.floofy.commons:gradle to v2.2.1.1 by @renovate-bot in https://github.com/Noelware/remi/pull/152

chore(deps): update dependency dev.floofy.commons:slf4j to v2.2.1.1 by @renovate-bot in https://github.com/Noelware/remi/pull/153

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.236 by @renovate-bot in https://github.com/Noelware/remi/pull/154

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.237 by @renovate-bot in https://github.com/Noelware/remi/pull/155

Full Changelog: https://github.com/Noelware/remi/compare/v0.3.2-beta...v0.4-beta
Source code(tar.gz)
Source code(zip)
v0.3.2-beta(Jul 8, 2022)
Fixes

fix(support-fs): use Files.createDirectories on the file's parent directory to ensure NoSuchFileExceptions occur.

Full Changelog: https://github.com/Noelware/remi/compare/v0.3.1-beta...v0.3.2-beta
Source code(tar.gz)
Source code(zip)
v0.3.1-beta(Jul 8, 2022)
Fixes

fix(support-s3): throwing NoSuchKeyException in #exists(Boolean) by @auguwu

fix(support-fs): throwing NoSuchFileException in #upload(String, java.io.InputStream, String?) by @auguwu

What's Changed

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.227 by @renovate-bot

Full Changelog: https://github.com/Noelware/remi/compare/v0.3-beta...v0.3.1-beta
Source code(tar.gz)
Source code(zip)
v0.3-beta(Jul 7, 2022)
Features

ETag support

When requesting a object or listing objects, you might want to see what the ETag for that file is. Remi adds the etag property in the Object so you don't have to calculate it!

Object creation date

The filesystem storage trailer now adds a new property to the object via the createdAt property. S3 and Minio don't support it.

Fetching a single object

You can fetch a single object via the fetch API supported in the main storage trailers. Read the documentation for more information.

Dependencies

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.225 by @renovate-bot

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.226 by @renovate-bot

chore(deps): update dependency com.google.cloud:google-cloud-storage to v2.9.2 by @renovate-bot

Full Changelog: https://github.com/Noelware/remi/compare/v0.2-beta...v0.3-beta
Source code(tar.gz)
Source code(zip)
v0.2-beta(Jul 4, 2022)
v0.2-beta

Additions

Add Object.path to determine where the file lives. The prefixes are determined by the storage trailer. (i.e, file://... -> Filesystem) by @auguwu

Add StorageTrailer.list to iterate objects via an prefix. The prefix can be ./ or ~/ on the filesystem storage trailer.

New properties: includeInputStream in StorageTrailer.list and StorageTrailer.listAll

If the input stream should be fetched. This is only applicable in the S3 or MinIO storage trailers. The filesystem one just ignores this. This is a property since re-fetching the input stream from the data source can be time-consuming if iterating over a lot of objects.

Dependency Updates

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.198 by @renovate-bot in https://github.com/Noelware/remi/pull/86

chore(deps): update dependency com.google.cloud:google-cloud-storage to v2.7.1 by @renovate-bot in https://github.com/Noelware/remi/pull/85

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.199 by @renovate-bot in https://github.com/Noelware/remi/pull/87

chore(deps): update dependency com.google.cloud:google-cloud-storage to v2.7.2 by @renovate-bot in https://github.com/Noelware/remi/pull/88

chore(deps): update dependency org.jetbrains.kotlinx:kotlinx-coroutines-bom to v1.6.2 by @renovate-bot in https://github.com/Noelware/remi/pull/89

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.201 by @renovate-bot in https://github.com/Noelware/remi/pull/90

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.202 by @renovate in https://github.com/Noelware/remi/pull/91

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.203 by @renovate in https://github.com/Noelware/remi/pull/92

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.204 by @renovate in https://github.com/Noelware/remi/pull/93

chore(deps): update dependency dev.floofy.commons:gradle to v2.1.1 by @renovate in https://github.com/Noelware/remi/pull/94

chore(deps): update dependency dev.floofy.commons:slf4j to v2.1.1 by @renovate in https://github.com/Noelware/remi/pull/95

chore(deps): update dependency com.diffplug.spotless:spotless-plugin-gradle to v6.7.0 by @renovate in https://github.com/Noelware/remi/pull/96

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.205 by @renovate in https://github.com/Noelware/remi/pull/97

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.206 by @renovate in https://github.com/Noelware/remi/pull/98

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.207 by @renovate in https://github.com/Noelware/remi/pull/99

chore(deps): update dependency com.google.cloud:google-cloud-storage to v2.8.0 by @renovate in https://github.com/Noelware/remi/pull/100

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.208 by @renovate in https://github.com/Noelware/remi/pull/101

chore(deps): update dependency com.diffplug.spotless:spotless-plugin-gradle to v6.7.1 by @renovate in https://github.com/Noelware/remi/pull/102

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.209 by @renovate in https://github.com/Noelware/remi/pull/103

chore(deps): update dependency com.diffplug.spotless:spotless-plugin-gradle to v6.7.2 by @renovate in https://github.com/Noelware/remi/pull/104

chore(deps): update dependency io.minio:minio to v8.4.2 by @renovate in https://github.com/Noelware/remi/pull/105

chore(deps): update dependency com.google.cloud:google-cloud-storage to v2.8.1 by @renovate in https://github.com/Noelware/remi/pull/106

chore(deps): update dependency io.kotest:kotest-bom to v5.3.1 by @renovate in https://github.com/Noelware/remi/pull/107

chore(deps): update dependency org.jetbrains.kotlinx:kotlinx-datetime to v0.3.3 by @renovate in https://github.com/Noelware/remi/pull/65

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.210 by @renovate in https://github.com/Noelware/remi/pull/108

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.211 by @renovate in https://github.com/Noelware/remi/pull/109

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.212 by @renovate in https://github.com/Noelware/remi/pull/110

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.213 by @renovate in https://github.com/Noelware/remi/pull/111

chore(deps): update dependency org.apache.tika:tika-core to v2.4.1 by @renovate in https://github.com/Noelware/remi/pull/112

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.214 by @renovate in https://github.com/Noelware/remi/pull/113

chore(deps): update dependency org.jetbrains.kotlinx:kotlinx-coroutines-bom to v1.6.3 by @renovate in https://github.com/Noelware/remi/pull/115

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.215 by @renovate in https://github.com/Noelware/remi/pull/116

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.216 by @renovate in https://github.com/Noelware/remi/pull/117

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.217 by @renovate in https://github.com/Noelware/remi/pull/118

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.218 by @renovate in https://github.com/Noelware/remi/pull/119

chore(deps): update dependency org.jetbrains.kotlinx:kotlinx-datetime to v0.4.0 by @renovate in https://github.com/Noelware/remi/pull/120

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.219 by @renovate in https://github.com/Noelware/remi/pull/121

chore(deps): update dependency io.kotest:kotest-bom to v5.3.2 by @renovate in https://github.com/Noelware/remi/pull/122

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.220 by @renovate in https://github.com/Noelware/remi/pull/123

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.221 by @renovate in https://github.com/Noelware/remi/pull/124

chore(deps): update dependency org.testcontainers:testcontainers to v1.17.3 by @renovate in https://github.com/Noelware/remi/pull/125

chore(deps): update dependency com.google.cloud:google-cloud-storage to v2.9.0 by @renovate in https://github.com/Noelware/remi/pull/126

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.222 by @renovate in https://github.com/Noelware/remi/pull/127

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.223 by @renovate in https://github.com/Noelware/remi/pull/128

chore(deps): update dependency software.amazon.awssdk:s3 to v2.17.224 by @renovate in https://github.com/Noelware/remi/pull/130

Full Changelog: https://github.com/Noelware/remi/compare/v0.1.5-beta...v0.2-beta
Source code(tar.gz)
Source code(zip)
v0.1.5-beta(May 24, 2022)
This release basically cleans up the filesystem storage trailer and dependency updates.

Dependency Updates

chore(deps): upgrade software.amazon.awssdk:s3 from v2.17.191 to v2.17.197 by @renovate-bot

chore(deps): upgrade org.testcontainers:testcontainers from v1.17.1 to v1.17.2 by @renovate-bot

Full Changelog: https://github.com/Noelware/remi/compare/v0.1.4-beta.4...v0.1.5-beta
Source code(tar.gz)
Source code(zip)
v0.1.4-beta.4(May 15, 2022)
Additions

support-fs: Added FilesystemStorageTrailer.normalizePath function by @auguwu

Dependency Updates

chore(deps): update dependency io.kotest:kotest-bom from v5.2.3 to v5.3.0 by @renovate-bot

chore(deps): update dependency org.apache.tika:tika-core from v2.3.0 to v2.4.0 by @renovate-bot

chore(deps): update dependency software.amazon.awssdk:s3 from v2.17.181 to v2.17.191 by @renovate-bot

chore(deps): update Gradle plugin com.diffplug.spotless:spotless-plugin-gradle from v6.5.2 to v6.6.1 by @renovate-bot

chore(deps): update dependency io.minio:minio from v8.3.9 to v8.4.1

chore(deps): update dependency org.jetbrains.kotlinx:kotlinx-serialization-bom from v1.3.2 to v1.3.3

Full Changelog: https://github.com/Noelware/remi/compare/v0.1.4-beta.3...v0.1.4-beta.4
Source code(tar.gz)
Source code(zip)
v0.1.4-beta.3(May 2, 2022)
Fixes

fix(s3): do not use AWS SDK BOM to do dependency updates by @auguwu

Dependency Updates

chore(deps): update dependency software.amazon.awssdk:s3 from v2.17.180 to v2.17.181 by @renovate-bot

Full Changelog: https://github.com/Noelware/remi/compare/v0.1.4-beta.2...v0.1.4-beta.3
Source code(tar.gz)
Source code(zip)
v0.1.4-beta.2(Apr 29, 2022)

Full Changelog: https://github.com/Noelware/remi/compare/v0.1.4-beta...v0.1.4-beta.2
Source code(tar.gz)
Source code(zip)
v0.1.4-beta(Apr 26, 2022)
Fixes

fix(fs): return the file path from Trailer.listAll() operation by @auguwu

fix(s3): do not paginate in Trailer.listAll() operation by @auguwu

Full Changelog: https://github.com/Noelware/remi/compare/v0.1.3-beta...v0.1.4-beta
Source code(tar.gz)
Source code(zip)
v0.1.3-beta(Apr 26, 2022)
Additions

feat(s3): add new config options: enforcePathAccessStyle and enableSignerV4Requests by @auguwu

Fixes

fix(filesystem): include directories (minus config.directory) in file path names by @auguwu

Dependency Updates

chore(deps): upgrade Dokka plugin from 1.6.20 -> 1.6.21 by @renovate-bot

chore(deps): upgrade software.amazon.awssdk:bom from 2.17.176 -> 2.17.177 by @renovate-bot

Full Changelog: https://github.com/Noelware/remi/compare/v0.1.2-beta...v0.1.3-beta
Source code(tar.gz)
Source code(zip)
v0.1.2-beta(Apr 25, 2022)
New Modules

remi-support-minio: Fully compatible module to work with MinIO.

Fixes

fix(s3): do not recursively call GetObject so retries don't occur by @auguwu (closes #55)

Additions

feat: Added extension method StorageTrailer#figureContentType to figure out the content type by @auguwu

feat: Added new constant: CHECK_WITH by @auguwu

Commit History: https://github.com/Noelware/remi/compare/v0.1.1-beta...v0.1.2-beta
Source code(tar.gz)
Source code(zip)
v0.1.1-beta(Apr 24, 2022)
Additions

feat: contentType in the Object class is now properly typed. by @auguwu

feat: new properties to Object: inputStream and original by @auguwu

feat: new method: Object.toInputStream by @auguwu

Dependency Updates

chore: upgrade software.amazon.awssdk:bom from 2.17.173 to 2.17.176 by @renovate-bot

chore: upgrade com.diffplug.spotless:spotless-plugin-gradle from 6.4.2 to 6.5.0 by @renovate-bot

chore: upgrade Kotlin compiler from 1.6.20 to 1.6.21 by @auguwu

Commit History: https://github.com/Noelware/remi/compare/v0.1.0-beta...v0.1.1-beta
Source code(tar.gz)
Source code(zip)
v0.1.0-beta(Apr 18, 2022)
:tada: We are now in beta!!! :tada:

Additions

feat: add in StorageTrailer.listAll/0 by @auguwu

Dependency Updates

chore: upgrade Gradle from 7.3.2 to 7.4.2 by @renovate-bot

chore: upgrade Kotlin (compiler, serialization plugin) from 1.6.10 to 1.6.20 by @renovate-bot

chore: upgrade org.jetbrains.kotlinx:kotlinx-serialization-bom from 1.3.1 to 1.3.2 by @renovate-bot

chore: upgrade org.jetbrains.kotlinx:kotlinx-coroutines-bom from 1.6.0-native-mt to 1.6.1 by @renovate-bot

chore: upgrade software.amazon.awssdk:bom from 2.17.133 to v2.17.172 by @renovate-bot

chore: upgrade io.kotest:kotest-bom from 5.0.3 to 5.2.3 by @renovate-bot

chore: upgrade org.jetbrains.dokka:dokka-gradle-plugin from 1.6.10 to 1.6.20 by @renovate-bot

chore: upgrade actions/setup-java from v2 to v3 by @renovate-bot

chore: upgrade actions/checkout from v2 to v3 by @renovate-bot

Commit History: https://github.com/Noelware/remi/compare/v0.0.4-SNAPSHOT...v0.1.0-beta
Source code(tar.gz)
Source code(zip)
v0.0.4-SNAPSHOT(Feb 24, 2022)

v0.0.4-SNAPSHOT

This release contains improvements to the Amazon S3 provider, that's pretty much it! And it unit tests the S3 provider to check for issues.
Source code(tar.gz)
Source code(zip)
v0.0.3-SNAPSHOT(Feb 23, 2022)
Fixes

Modules are actually published as JARs now

Source code(tar.gz)
Source code(zip)

v0.0.2-SNAPSHOT(Feb 21, 2022)

Additions

[remi-core] Added a new interface method to StorageTrailer: StorageTrailer#upload by @auguwu
[remi-support-fs] Added ./ and ~/ support for file paths by @auguwu

Full Changelog: https://github.com/Noelware/remi/compare/v0.0.1-SNAPSHOT...v0.0.2-SNAPSHOT

Installation

Gradle

Kotlin DSL

repositories {
    // If you're using the Noel Gradle Utils package, you can use the
    // `noelware` extension
    maven {
        url = uri("https://maven.noelware.org/snapshots")
    }
}

dependencies {
    // If you're using the Noel Gradle Utils package, you can use
    // the `noelware` extension to automatically prefix `org.noelware.<module>`
    // in the dependency declaration
    implementation("org.noelware.remi:remi-<module_name>:<version>")
}

Groovy DSL

repositories {
    maven {
        url "https://maven.noelware.org/snapshots"
    }
}

dependencies {
    implementation "org.noelware.remi:remi-<module_name>:<version>"
}

Maven

Declare the Noelware Snapshots Maven repository under the <repositories> chain:

<repositories>
    <repository>
        <id>noelware-maven</id>
        <url>https://maven.noelware.org/snapshots</url>
    </repository>
</repositories>

Now declare the dependency you want under the <dependencies> chain:

<dependencies>
    <dependency>
        <groupId>org.noelware.remi</groupId>
        <artifactId>remi-{{NAME}}</artifactId>
        <version>{{VERSION}}</version>
        <type>pom</type>
    </dependency>
</dependencies>

Source code(tar.gz)
Source code(zip)

v0.0.1-SNAPSHOT(Feb 20, 2022)

This is a snapshot release of Remi for testing purposes. You can install the library using the maven.noelware.org/snapshots repository:

Gradle

Kotlin DSL

repositories {
    // If you're using the Noel Gradle Utils package, you can use the
    // `noelware` extension
    maven {
        url = uri("https://maven.noelware.org/snapshots")
    }
}

dependencies {
    // If you're using the Noel Gradle Utils package, you can use
    // the `noelware` extension to automatically prefix `org.noelware.<module>`
    // in the dependency declaration
    implementation("org.noelware.remi:remi-<module_name>:<version>")
}

Groovy DSL

repositories {
    maven {
        url "https://maven.noelware.org/snapshots"
    }
}

dependencies {
    implementation "org.noelware.remi:remi-<module_name>:<version>"
}

Maven

Declare the Noelware Snapshots Maven repository under the <repositories> chain:

<repositories>
    <repository>
        <id>noelware-maven</id>
        <url>https://maven.noelware.org/snapshots</url>
    </repository>
</repositories>

Now declare the dependency you want under the <dependencies> chain:

<dependencies>
    <dependency>
        <groupId>org.noelware.remi</groupId>
        <artifactId>remi-{{NAME}}</artifactId>
        <version>{{VERSION}}</version>
        <type>pom</type>
    </dependency>
</dependencies>

Source code(tar.gz)
Source code(zip)