MaigoLabs/refray
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[O] UX

Browse Source
This commit is contained in:
Azalea Gui
2026-05-08 16:24:23 +00:00
parent 3d73f20c1f
commit 018f1f12d5
5 changed files with 188 additions and 236 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+56 -63
View File
@@ -13,6 +13,9 @@ Created becasue github is so unusable and [unreliable](https://red-squares.cian.
Supported platforms: GitHub, GitLab, Gitea, Forgejo
> [!NOTE]
> Meow
## Install
### Option 1. Install from source
@@ -36,7 +39,9 @@ docker compose up -d --build
docker compose run --rm --entrypoint nano refray /data/config/refray/config.toml
```
## Configure
## Usage
### 1. Configure
Run the interactive configuration wizard:
@@ -44,7 +49,45 @@ Run the interactive configuration wizard:
refray config
```
## One-time Sync
<details><summary>Example Config</summary>
```toml
[[sites]]
name = "github"
provider = "github"
base_url = "https://github.com"
token = { env = "GITHUB_TOKEN" }
[[sites]]
name = "gitea"
provider = "gitea"
base_url = "https://gitea.example.com"
token = { env = "GITEA_TOKEN" }
[[mirrors]]
name = "personal"
sync_visibility = "all"
repo_whitelist = ["^important-"]
repo_blacklist = ["-archive$"]
create_missing = true
visibility = "private"
allow_force = false
conflict_resolution = "auto_rebase_pull_request"
[[mirrors.endpoints]]
site = "github"
kind = "user"
namespace = "hykilpikonna"
[[mirrors.endpoints]]
site = "gitea"
kind = "user"
namespace = "azalea"
```
</details>
### 2. One-time Sync
Run all configured mirror groups:
@@ -88,54 +131,40 @@ refray sync --jobs 8
</details>
## Service & Webhooks
### 3. Service & Webhooks
You can run `refray` as a service that listens for webhook events and runs full sync periodically. This is the recommended way to run `refray`.
If you want to use webhooks, you need to expose port 8787 to a public URL that can be accessed by the git provider (e.g. using port forwarding, reverse proxy, or cloudflare tunnel).
> [!NOTE]
> If you want to use webhooks, you need to expose port 8787 to a public URL that can be accessed by the git provider.<br>
> This can be done using e.g. port forwarding, reverse proxy, cloudflare tunnel, or tailscale funnel.
Start the receiver:
Start the service:
```sh
refray serve
```
Expose that listener with your reverse proxy or tunnel, then install repository webhooks. If `[webhook]` is configured, the URL and secret can come from config:
Install webhooks on all repos:
```sh
refray webhook install
```
Useful install filters:
```sh
refray webhook install \
--url https://mirror.example.com/webhook \
--secret-env REFRAY_WEBHOOK_SECRET \
--group personal \
--repo-pattern '^important-'
```
The receiver accepts `POST /` and `POST /webhook`. It verifies GitHub/Gitea HMAC SHA-256 signatures and GitLab webhook tokens, then queues `refray sync --group <group> --repo-pattern '^<repo>$'` internally. Duplicate events for the same group/repo are coalesced while a job is queued or running. Sync jobs are serialized inside the receiver so the local ref and failure caches stay consistent.
When `[webhook].install = true`, normal `refray sync` also checks webhook installation status and installs missing webhooks for repositories that have not been recorded yet. Installation status is stored in `webhook-state.toml` under the work directory.
Webhook install uses `[webhook].url` and `[webhook].secret` from your config.
To uninstall webhooks previously installed by `refray`:
> [!WARNING]
> If you want to stop using `refray`, make sure you run this! Otherwise, all of your repos will keep trying to send webhooks to the URL.
```sh
refray webhook uninstall
```
Manual `webhook uninstall` checks repositories on the provider instead of trusting only local state. To uninstall one repository exactly:
```sh
refray webhook uninstall important-repo
```
To move installed hooks to a new public URL, use `webhook update`. It removes hooks matching the current configured `[webhook].url`, installs the new URL, updates `[webhook].url` in the config, and refreshes local webhook state:
```sh
refray webhook update --url https://new.example.com/webhook
refray webhook update https://new.example.com/webhook
```
Serve can also run periodic full syncs. The interval can be configured in `[webhook].full_sync_interval_minutes` or overridden at startup:
@@ -183,42 +212,6 @@ Branch deletion follows the same rule at branch scope: if a branch existed on ev
Tags are fetched into provider-specific cache refs and pushed only when the tag object agrees across providers or exists on one side. Divergent tags are skipped and reported. Tag deletion is not propagated.
## Example Config
```toml
[[sites]]
name = "github"
provider = "github"
base_url = "https://github.com"
token = { env = "GITHUB_TOKEN" }
[[sites]]
name = "gitea"
provider = "gitea"
base_url = "https://gitea.example.com"
token = { env = "GITEA_TOKEN" }
[[mirrors]]
name = "personal"
sync_visibility = "all"
repo_whitelist = ["^important-"]
repo_blacklist = ["-archive$"]
create_missing = true
visibility = "private"
allow_force = false
conflict_resolution = "auto_rebase_pull_request"
[[mirrors.endpoints]]
site = "github"
kind = "user"
namespace = "hykilpikonna"
[[mirrors.endpoints]]
site = "gitea"
kind = "user"
namespace = "azalea"
```
## Testing
Run the normal, non-destructive test suite:
src/main.rs
+30 -42
View File
@@ -89,50 +89,24 @@ enum WebhookCommand {
#[derive(Args, Debug)]
struct WebhookInstallCommand {
#[arg(value_name = "REPO", conflicts_with = "repo_pattern")]
repo: Option<String>,
#[arg(long, value_name = "URL")]
url: Option<String>,
#[arg(long, conflicts_with = "secret_env")]
secret: Option<String>,
#[arg(long, value_name = "ENV", conflicts_with = "secret")]
secret_env: Option<String>,
#[arg(long, value_name = "NAME")]
group: Option<String>,
#[arg(long, value_name = "REGEX")]
repo_pattern: Option<String>,
#[arg(long)]
dry_run: bool,
#[arg(long, value_name = "PATH")]
work_dir: Option<PathBuf>,
#[arg(long, default_value_t = DEFAULT_JOBS, value_name = "N")]
jobs: usize,
}
#[derive(Args, Debug)]
struct WebhookUninstallCommand {
#[arg(value_name = "REPO")]
repo: Option<String>,
#[arg(long, value_name = "URL")]
url: Option<String>,
#[arg(long, value_name = "NAME")]
group: Option<String>,
#[arg(long)]
dry_run: bool,
#[arg(long, value_name = "PATH")]
work_dir: Option<PathBuf>,
#[arg(long, default_value_t = DEFAULT_JOBS, value_name = "N")]
jobs: usize,
}
#[derive(Args, Debug)]
struct WebhookUpdateCommand {
#[arg(long, value_name = "URL")]
#[arg(value_name = "URL")]
url: String,
#[arg(long, conflicts_with = "secret_env")]
secret: Option<String>,
#[arg(long, value_name = "ENV", conflicts_with = "secret")]
secret_env: Option<String>,
#[arg(long)]
dry_run: bool,
#[arg(long, value_name = "PATH")]
@@ -199,33 +173,28 @@ fn main() -> Result<()> {
}
Command::Webhook(WebhookCommand::Install(command)) => {
let config = load_config(&config_path)?;
let secret = resolve_webhook_secret(&config, command.secret, command.secret_env)?;
let url = resolve_webhook_url(&config, command.url)?;
let secret = resolve_config_webhook_secret(&config)?;
let url = resolve_config_webhook_url(&config)?;
install_webhooks(
&config,
WebhookInstallOptions {
url,
secret,
group: command.group,
repo: command.repo,
repo_pattern: command.repo_pattern,
dry_run: command.dry_run,
work_dir: command.work_dir,
work_dir: None,
jobs: command.jobs,
},
)
}
Command::Webhook(WebhookCommand::Uninstall(command)) => {
let config = load_config(&config_path)?;
let url = resolve_webhook_url(&config, command.url)?;
let url = resolve_config_webhook_url(&config)?;
uninstall_webhooks(
&config,
WebhookUninstallOptions {
url,
group: command.group,
repo: command.repo,
dry_run: command.dry_run,
work_dir: command.work_dir,
work_dir: None,
jobs: command.jobs,
},
)
@@ -239,7 +208,7 @@ fn main() -> Result<()> {
.ok_or_else(|| {
anyhow::anyhow!("configure [webhook] before running webhook update")
})?;
let secret = resolve_webhook_secret(&config, command.secret, command.secret_env)?;
let secret = resolve_config_webhook_secret(&config)?;
update_webhooks(
&config,
WebhookUpdateOptions {
@@ -261,9 +230,26 @@ fn main() -> Result<()> {
}
fn load_config(path: &Path) -> Result<Config> {
if !path.exists() {
anyhow::bail!(
"config not found at {}. Run `refray config` first to create one.",
path.display()
);
}
Config::load(path).with_context(|| format!("failed to load config at {}", path.display()))
}
fn resolve_config_webhook_secret(config: &Config) -> Result<String> {
config
.webhook
.as_ref()
.map(|webhook| webhook.secret())
.transpose()?
.ok_or_else(|| {
anyhow::anyhow!("configure [webhook].secret before running webhook commands")
})
}
fn resolve_webhook_secret(
config: &Config,
value: Option<String>,
@@ -283,10 +269,12 @@ fn resolve_webhook_secret(
}
}
fn resolve_webhook_url(config: &Config, value: Option<String>) -> Result<String> {
value
.or_else(|| config.webhook.as_ref().map(|webhook| webhook.url.clone()))
.ok_or_else(|| anyhow::anyhow!("pass --url or configure [webhook].url"))
fn resolve_config_webhook_url(config: &Config) -> Result<String> {
config
.webhook
.as_ref()
.map(|webhook| webhook.url.clone())
.ok_or_else(|| anyhow::anyhow!("configure [webhook].url before running webhook commands"))
}
fn set_config_webhook_url(config: &mut Config, url: String) {
src/webhook.rs
-42
View File
@@ -39,9 +39,6 @@ pub struct ServeOptions {
pub struct WebhookInstallOptions {
pub url: String,
pub secret: String,
pub group: Option<String>,
pub repo: Option<String>,
pub repo_pattern: Option<String>,
pub dry_run: bool,
pub work_dir: Option<PathBuf>,
pub jobs: usize,
@@ -50,8 +47,6 @@ pub struct WebhookInstallOptions {
#[derive(Clone, Debug)]
pub struct WebhookUninstallOptions {
pub url: String,
pub group: Option<String>,
pub repo: Option<String>,
pub dry_run: bool,
pub work_dir: Option<PathBuf>,
pub jobs: usize,
@@ -183,21 +178,8 @@ pub fn install_webhooks(config: &Config, options: WebhookInstallOptions) -> Resu
}
let work_dir = options.work_dir.clone().unwrap_or_else(default_work_dir);
let state = Arc::new(Mutex::new(load_webhook_state(&work_dir)?));
let repo_pattern = options
.repo_pattern
.as_deref()
.map(regex::Regex::new)
.transpose()
.with_context(|| "invalid --repo-pattern regex")?;
for mirror in &config.mirrors {
if options
.group
.as_ref()
.is_some_and(|group| group != &mirror.name)
{
continue;
}
crate::logln!();
crate::logln!(
"{} {}",
@@ -222,15 +204,6 @@ pub fn install_webhooks(config: &Config, options: WebhookInstallOptions) -> Resu
.filter(|repo| mirror.sync_visibility.matches_private(repo.private))
.filter(|repo| repo_filter.matches(&repo.name))
{
if options.repo.as_ref().is_some_and(|name| name != &repo.name) {
continue;
}
if repo_pattern
.as_ref()
.is_some_and(|pattern| !pattern.is_match(&repo.name))
{
continue;
}
tasks.push(WebhookInstallTask {
site: site.clone(),
group: mirror.name.clone(),
@@ -262,13 +235,6 @@ pub fn uninstall_webhooks(config: &Config, options: WebhookUninstallOptions) ->
let mut state = load_webhook_state(&work_dir)?;
let mut tasks = Vec::new();
for mirror in &config.mirrors {
if options
.group
.as_ref()
.is_some_and(|group| group != &mirror.name)
{
continue;
}
crate::logln!();
crate::logln!(
"{} {}",
@@ -287,9 +253,6 @@ pub fn uninstall_webhooks(config: &Config, options: WebhookUninstallOptions) ->
.list_repos(endpoint)
.with_context(|| format!("failed to list repos for {}", endpoint.label()))?;
for repo in repos {
if options.repo.as_ref().is_some_and(|name| name != &repo.name) {
continue;
}
tasks.push(WebhookUninstallTask {
group: mirror.name.clone(),
site: site.clone(),
@@ -329,8 +292,6 @@ pub fn update_webhooks(config: &Config, options: WebhookUpdateOptions) -> Result
config,
WebhookUninstallOptions {
url: options.old_url.clone(),
group: None,
repo: None,
dry_run: options.dry_run,
work_dir: options.work_dir.clone(),
jobs: options.jobs,
@@ -343,9 +304,6 @@ pub fn update_webhooks(config: &Config, options: WebhookUpdateOptions) -> Result
WebhookInstallOptions {
url: options.new_url,
secret: options.secret,
group: None,
repo: None,
repo_pattern: None,
dry_run: options.dry_run,
work_dir: options.work_dir,
jobs: options.jobs,
tests/e2e/sequential.rs
+2 -22
View File
@@ -646,33 +646,13 @@ namespace = "{}"
self.seed_all_main(&repo, "webhook base", 1_700_001_601)?;
self.sync(["--repo-pattern", &exact_pattern(&repo)])?;
self.refray([
"webhook",
"install",
"--dry-run",
"--repo-pattern",
&exact_pattern(&repo),
"--url",
"https://example.invalid/webhook",
"--secret",
WEBHOOK_SECRET,
])?;
self.refray([
"webhook",
"uninstall",
&repo,
"--dry-run",
"--url",
"https://example.invalid/webhook",
])?;
self.refray(["webhook", "install", "--dry-run"])?;
self.refray(["webhook", "uninstall", "--dry-run"])?;
self.refray([
"webhook",
"update",
"--dry-run",
"--url",
"https://example.invalid/new-webhook",
"--secret",
WEBHOOK_SECRET,
])?;
let listener = TcpListener::bind("127.0.0.1:0")?;
tests/unit/cli.rs
+94 -61
View File
@@ -85,98 +85,88 @@ fn cli_accepts_webhook_serve() {
#[test]
fn cli_accepts_webhook_install() {
let cli = Cli::try_parse_from([
"refray",
"webhook",
"install",
"repo-one",
"--url",
"https://mirror.example.test/webhook",
"--secret",
"secret",
"--group",
"sync-1",
"--jobs",
"6",
])
.unwrap();
let cli = Cli::try_parse_from(["refray", "webhook", "install", "--jobs", "6"]).unwrap();
let Command::Webhook(WebhookCommand::Install(args)) = cli.command else {
panic!("parsed unexpected command");
};
assert_eq!(
args.url,
Some("https://mirror.example.test/webhook".to_string())
);
assert_eq!(args.secret, Some("secret".to_string()));
assert_eq!(args.group, Some("sync-1".to_string()));
assert_eq!(args.repo, Some("repo-one".to_string()));
assert_eq!(args.repo_pattern, None);
assert_eq!(args.jobs, 6);
}
#[test]
fn cli_accepts_webhook_install_repo_pattern() {
let cli = Cli::try_parse_from([
fn cli_rejects_removed_webhook_install_args() {
for args in [
["refray", "webhook", "install", "repo-one"].as_slice(),
[
"refray",
"webhook",
"install",
"--url",
"https://mirror.example.test/webhook",
"--secret",
"secret",
"--repo-pattern",
"^repo$",
])
.unwrap();
let Command::Webhook(WebhookCommand::Install(args)) = cli.command else {
panic!("parsed unexpected command");
};
assert_eq!(args.repo, None);
assert_eq!(args.repo_pattern, Some("^repo$".to_string()));
]
.as_slice(),
["refray", "webhook", "install", "--group", "sync-1"].as_slice(),
["refray", "webhook", "install", "--work-dir", "/tmp/refray"].as_slice(),
["refray", "webhook", "install", "--secret", "secret"].as_slice(),
[
"refray",
"webhook",
"install",
"--secret-env",
"WEBHOOK_SECRET",
]
.as_slice(),
["refray", "webhook", "install", "--repo-pattern", "^repo$"].as_slice(),
] {
assert!(Cli::try_parse_from(args).is_err());
}
}
#[test]
fn cli_accepts_webhook_uninstall() {
let cli = Cli::try_parse_from([
"refray",
"webhook",
"uninstall",
"repo-one",
"--url",
"https://mirror.example.test/webhook",
"--group",
"sync-1",
"--dry-run",
"--jobs",
"3",
])
let cli = Cli::try_parse_from(["refray", "webhook", "uninstall", "--dry-run", "--jobs", "3"])
.unwrap();
let Command::Webhook(WebhookCommand::Uninstall(args)) = cli.command else {
panic!("parsed unexpected command");
};
assert_eq!(
args.url,
Some("https://mirror.example.test/webhook".to_string())
);
assert_eq!(args.group, Some("sync-1".to_string()));
assert_eq!(args.repo, Some("repo-one".to_string()));
assert!(args.dry_run);
assert_eq!(args.jobs, 3);
}
#[test]
fn cli_rejects_removed_webhook_uninstall_args() {
for args in [
["refray", "webhook", "uninstall", "repo-one"].as_slice(),
[
"refray",
"webhook",
"uninstall",
"--url",
"https://mirror.example.test/webhook",
]
.as_slice(),
["refray", "webhook", "uninstall", "--group", "sync-1"].as_slice(),
[
"refray",
"webhook",
"uninstall",
"--work-dir",
"/tmp/refray",
]
.as_slice(),
] {
assert!(Cli::try_parse_from(args).is_err());
}
}
#[test]
fn cli_accepts_webhook_update() {
let cli = Cli::try_parse_from([
"refray",
"webhook",
"update",
"--url",
"https://new.example.test/webhook",
"--secret-env",
"WEBHOOK_SECRET",
"--jobs",
"5",
])
@@ -186,10 +176,43 @@ fn cli_accepts_webhook_update() {
panic!("parsed unexpected command");
};
assert_eq!(args.url, "https://new.example.test/webhook");
assert_eq!(args.secret_env, Some("WEBHOOK_SECRET".to_string()));
assert_eq!(args.jobs, 5);
}
#[test]
fn cli_rejects_removed_webhook_update_secret_args() {
for args in [
[
"refray",
"webhook",
"update",
"--url",
"https://new.example.test/webhook",
]
.as_slice(),
[
"refray",
"webhook",
"update",
"https://new.example.test/webhook",
"--secret",
"secret",
]
.as_slice(),
[
"refray",
"webhook",
"update",
"https://new.example.test/webhook",
"--secret-env",
"WEBHOOK_SECRET",
]
.as_slice(),
] {
assert!(Cli::try_parse_from(args).is_err());
}
}
#[test]
fn cli_rejects_scoped_webhook_update() {
let result = Cli::try_parse_from([
@@ -197,9 +220,19 @@ fn cli_rejects_scoped_webhook_update() {
"webhook",
"update",
"repo-one",
"--url",
"https://new.example.test/webhook",
]);
assert!(result.is_err());
}
#[test]
fn missing_config_error_asks_user_to_run_config() {
let temp = tempfile::tempdir().unwrap();
let path = temp.path().join("config.toml");
let error = load_config(&path).unwrap_err().to_string();
assert!(error.contains("config not found at"));
assert!(error.contains("Run `refray config` first"));
}