You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[update] Use Rclone to Sync Files to Linode Object Storage (linode#6465)
* [update] Use Rclone to Sync Files to Linode Object Storage
_ remove the numbers for the options and reword
* Fixed formatting
* Tech/copy edit
---------
Co-authored-by: Matt Wildman <matt@wildman.online>
title: "Use Rclone to Sync Files to Linode Object Storage"
@@ -28,83 +29,120 @@ For years, [rsync](/docs/guides/introduction-to-rsync/) has been the go-to backu
28
29
29
30
The Rclone installation process is the same on Linux and macOS. Log into either platform and open a terminal window. From the terminal, issue the following command:
30
31
31
-
curl https://rclone.org/install.sh | sudo bash
32
+
```command
33
+
curl https://rclone.org/install.sh | sudo bash
34
+
```
32
35
33
-
{{< note respectIndent=false >}}
36
+
{{< note >}}
34
37
To install cURL on an Ubuntu system, use the following command:
35
38
36
-
sudo apt-get install curl -y
39
+
```command
40
+
sudo apt-get install curl -y
41
+
```
37
42
38
43
To install cURL on a macOS system use the following command:
You should see the version of Rclone installed, as well as some information about the platform on which it was installed.
49
58
50
59
## Configure Rclone
51
60
52
61
Before you configure Rclone, [create a new Linode bucket](/docs/products/storage/object-storage/guides/manage-buckets/) and then [generate an associated Access Key](/docs/products/storage/object-storage/guides/access-keys/) for that bucket. You can also create a new access key for an existing bucket. When you create an access key, you are given the **Access Key** and the **Secret Key**. Make sure to copy both of those strings, because you need them when configuring Rclone.
53
62
54
-
1. Once you have created the Linode bucket and have the access keys, set up a new configuration with the following command:
63
+
1.Once you have created the Linode bucket and have the access keys, set up a new configuration with the following command:
55
64
56
-
rclone config
65
+
```command
66
+
rclone config
67
+
```
57
68
58
-
1.When prompted, configure a **New Remote** by typing **n**.
69
+
1. Configure a **New Remote** by typing `n`.
59
70
60
-
1.Use a human-readable name as the configuration name.
71
+
1. Next, enter a name to use your new remote.
61
72
62
-
1.Then, select your cloud storage service. In this case, select AWS S3 Compliant Storage Provider (option 4), followed by selecting any other S3 compatible provider (option 14).
73
+
1. When prompted for the type of storage, selecttheoption that corresponds with **S3** (*"AWS S3 Compliant Storage Providers including..."*).
63
74
64
-
1. Get AWS credentials. For this step, type `2` and then type your **Access Key** string (for `access_key_id`) and then paste your Secret Key for `secret_access_key`.
1. Then, select**Other** (*"Any other S3 compatible provider"*) when choosing your provider.
67
81
68
-
1. For the endpoint, type the address for your Linode bucket, such as `rclone.us-east-1.linodeobjects.com`.
82
+
```output
83
+
27 / Any other S3 compatible provider
84
+
\ (Other)
85
+
```
69
86
70
-
1.Leave `location_constraint` blank.
87
+
1. You are then asked how you'd like to provide your AWS credentials. For this, select **false** to manually enter your access key and secret.
71
88
72
-
1. Select the `acl` you'd like to use. If this is a personal bucket, type `1` for private. If this is a team bucket, you might select `public-read` (option 2) or `authentication-read` (option 4).
89
+
```output
90
+
1 / Enter AWS credentials in the next step.
91
+
\ (false)
92
+
```
73
93
74
-
1. Hit **enter** to opt-out of editing the advanced configuration file.
94
+
As part of this step, you'll be requested to enter an access key and secret key for our Object Storage service. If you do not have this information, you can [generate a new access key](/docs/products/storage/object-storage/guides/access-keys/#create-an-access-key).
75
95
76
-
1.Verify the configuration and, if it is correct, type **y** to save the options.
96
+
1. When prompted, leave the `region` section blank.
77
97
78
-
1. Type **q** to quit the configuration tool.
98
+
1. For the endpoint, type the URL that corresponds with the region your buckets are located within. Review [Cluster URL (S3 Endpoint)](/docs/products/storage/object-storage/guides/urls/#cluster-url-s3-endpoint) for a full list.
99
+
100
+
1. Leave `location_constraint` blank.
101
+
102
+
1. Next, you are requested to enter the Access Control List (ACL) to use when creating buckets and objects. This is based on your preferences and how you intend to use your buckets. Review [Define Access and Permissions using ACLs (Access Control Lists)](/docs/products/storage/object-storage/guides/acls/) for more details about ACLs and the various permission levels. For a personal bucket (or if you are unsure), you may want to enter *private*.
103
+
104
+
1. You are then given the option for advanced configuration. For this tutorial, select*no* or just hit enter.
105
+
106
+
1. Finally, an output containing your configuration is displayed. Verify that this information looks correct and enter **y** to save it.
107
+
108
+
1. To finish and exit the configuration tool, enter **q** at the last prompt.
79
109
80
110
## Rclone Sync
81
111
82
112
You can sync a local directory to your Linode storage bucket. To do that, create a new bucket using the command below. The example command names the bucket `test` and the remote is named `Linode`. Replace these names with your own.
83
113
84
-
rclone mkdir Linode:test
114
+
```command
115
+
rclone mkdir Linode:test
116
+
```
85
117
86
118
When you open your Linode cloud manager, you now see a `test` object in your bucket.
87
119
88
120
When you want to sync your local`Documents` directory to that remote test object use the following command:
89
121
90
-
rclone sync Documents Linode:test
122
+
```command
123
+
rclone sync Documents Linode:test
124
+
```
91
125
92
126
Once the sync is complete, you now see all of the files in the local`Documents` directory, in your Linode Object Storage bucket.
93
127
94
128
## Rclone Copy
95
129
96
130
The `rclone copy`command copies the source to the destination but does not copy files from the destination to the source. Say you have a file named `testing.txt` and you want to copy it to the `test` bucket on your Linode Object Storage "remote". You can use the following command to accomplish this:
97
131
98
-
rclone copy testing Linode:test
132
+
```command
133
+
rclone copy testing Linode:test
134
+
```
99
135
100
136
Check out your `test` bucket and you now see the `testing.txt` file added.
101
137
102
138
## Rclone Mount
103
139
104
140
You can also mount a local directory to a Linode Object Storage bucket. The one caveat to this is that the local directory must be empty. You should also know that the `rclone mount`command does take a long time to complete. It's a known issue that writes to `rclone mount` are extremely slow. Because of this, your best bet is to use the `rclone sync` command instead. However, if you still want to mount a directory, the command is straightforward. To mount the local directory, `/home/example-user/LINODE` to a bucket named `DATA` on the Linode remote, use the following command:
105
141
106
-
rclone mount Linode:DATA /home/example-user/LINODE
142
+
```command
143
+
rclone mount Linode:DATA /home/example-user/LINODE
144
+
```
107
145
108
146
Once the directory is mounted, dump all of the necessary files into the local source and they are synced with the remote destination.
109
147
110
-
The command covered in this guide give you all the basics you need to use Rclone to sync and backup files to your Linode Object Storage buckets.
148
+
The command covered in this guide give you all the basics you need to use Rclone to sync and backup files to your Linode Object Storage buckets.
0 commit comments