How do I identify the Google Cloud Storage URI from my Google Developers Console?
When I attempt load data into BigQuery from Google Cloud Storage it asks for the Google Cloud Storage URI (gs://). I have reviewed all of your online support as well as stackoverflow and cannot find a way to identify the URL for my uploaded data via the browser based Google Developers Console. The only way I see to find the URL is via gsutil and I have not been able to get gsutil to work on my machine.
Is there a way to determine the URL via the browser based Google Developers Console?
google-cloud-platform google-bigquery google-cloud-storage
add a comment |
When I attempt load data into BigQuery from Google Cloud Storage it asks for the Google Cloud Storage URI (gs://). I have reviewed all of your online support as well as stackoverflow and cannot find a way to identify the URL for my uploaded data via the browser based Google Developers Console. The only way I see to find the URL is via gsutil and I have not been able to get gsutil to work on my machine.
Is there a way to determine the URL via the browser based Google Developers Console?
google-cloud-platform google-bigquery google-cloud-storage
Could you post (maybe a separate question) the problems you encountered trying to set up gsutil?
– jterrace
Aug 18 '14 at 23:28
yes, how did you upload your data? If it's small enough you can upload it straight to BigQuery
– Felipe Hoffa
Aug 19 '14 at 3:05
1
I uploaded data via the google developers console. The suggestion to try gs://bucket/file name worked. This was very helpful.
– Kelly
Aug 19 '14 at 4:36
add a comment |
When I attempt load data into BigQuery from Google Cloud Storage it asks for the Google Cloud Storage URI (gs://). I have reviewed all of your online support as well as stackoverflow and cannot find a way to identify the URL for my uploaded data via the browser based Google Developers Console. The only way I see to find the URL is via gsutil and I have not been able to get gsutil to work on my machine.
Is there a way to determine the URL via the browser based Google Developers Console?
google-cloud-platform google-bigquery google-cloud-storage
When I attempt load data into BigQuery from Google Cloud Storage it asks for the Google Cloud Storage URI (gs://). I have reviewed all of your online support as well as stackoverflow and cannot find a way to identify the URL for my uploaded data via the browser based Google Developers Console. The only way I see to find the URL is via gsutil and I have not been able to get gsutil to work on my machine.
Is there a way to determine the URL via the browser based Google Developers Console?
google-cloud-platform google-bigquery google-cloud-storage
google-cloud-platform google-bigquery google-cloud-storage
edited Sep 1 '18 at 8:22
Pat Myron
62211027
62211027
asked Aug 18 '14 at 23:14
KellyKelly
173135
173135
Could you post (maybe a separate question) the problems you encountered trying to set up gsutil?
– jterrace
Aug 18 '14 at 23:28
yes, how did you upload your data? If it's small enough you can upload it straight to BigQuery
– Felipe Hoffa
Aug 19 '14 at 3:05
1
I uploaded data via the google developers console. The suggestion to try gs://bucket/file name worked. This was very helpful.
– Kelly
Aug 19 '14 at 4:36
add a comment |
Could you post (maybe a separate question) the problems you encountered trying to set up gsutil?
– jterrace
Aug 18 '14 at 23:28
yes, how did you upload your data? If it's small enough you can upload it straight to BigQuery
– Felipe Hoffa
Aug 19 '14 at 3:05
1
I uploaded data via the google developers console. The suggestion to try gs://bucket/file name worked. This was very helpful.
– Kelly
Aug 19 '14 at 4:36
Could you post (maybe a separate question) the problems you encountered trying to set up gsutil?
– jterrace
Aug 18 '14 at 23:28
Could you post (maybe a separate question) the problems you encountered trying to set up gsutil?
– jterrace
Aug 18 '14 at 23:28
yes, how did you upload your data? If it's small enough you can upload it straight to BigQuery
– Felipe Hoffa
Aug 19 '14 at 3:05
yes, how did you upload your data? If it's small enough you can upload it straight to BigQuery
– Felipe Hoffa
Aug 19 '14 at 3:05
1
1
I uploaded data via the google developers console. The suggestion to try gs://bucket/file name worked. This was very helpful.
– Kelly
Aug 19 '14 at 4:36
I uploaded data via the google developers console. The suggestion to try gs://bucket/file name worked. This was very helpful.
– Kelly
Aug 19 '14 at 4:36
add a comment |
6 Answers
6
active
oldest
votes
The path should be gs://<bucket_name>/<file_path_inside_bucket>
.
15
Ridiculous that I had to find this answer on SO rather than in Google's own docs. Thank you @nmore!
– Stan James
Sep 14 '16 at 17:25
It's regrettable that this question did not get favorited
– chrisfs
Jan 22 '18 at 2:18
add a comment |
Follow the following steps :
1. Go to GCS
2. Go into the folder in which the file has been uploaded
3. On the top you can see overview option
4. You can see there will be Link URL and link for GSUtil
add a comment |
To answer this question more information is needed. Did you already load your data into GCS?
If not, the easiest would be to go to the project console, click on project, and Storage -> Cloud Storage -> Storage browser.
You can create buckets there and upload files to the bucket.
Then the files will be found at gs://<bucket_name>/<file_path_inside_bucket>
as @nmore says.
add a comment |
Couldn't find a direct way to get the url. But found an indirect way and below are the steps:
- Go to GCS
- Go into the folder in which the file has been uploaded
- Click on the three dots at the right end of your file's row
- Click rename
- Click on gsutil equivalent link
- Copy the url alone
add a comment |
If you need help on subdirectories, check this out on https://cloud.google.com/storage/docs/gsutil/addlhelp/HowSubdirectoriesWork
And https://cloud.google.com/storage/images/gsutil-subdirectories-thumb.png, if you need to see how gsutil provides a hierarchical view of objects in a bucket.
add a comment |
Retrieving the Google Cloud Storage URI
To create an external table using a Google Cloud Storage data source, you must provide the Cloud Storage URI.
The Cloud Storage URI comprises your bucket name and your object (filename). For example, if the Cloud Storage bucket is named mybucket and the data file is named myfile.csv, the bucket URI would be gs://mybucket/myfile.csv. If your data is separated into multiple files you can use a wildcard in the URI. For more information, see Cloud Storage Request URIs.
BigQuery does not support source URIs that include multiple consecutive slashes after the initial double slash. Cloud Storage object names can contain multiple consecutive slash ("/") characters. However, BigQuery converts multiple consecutives slashes into a single slash. For example, the following source URI, though valid in Cloud Storage, does not work in BigQuery: gs://[BUCKET]/my//object//name.
To retrieve the Cloud Storage URI:
Open the Cloud Storage web UI.
CLOUD STORAGE WEB UI
Browse to the location of the object (file) that contains the source data.
At the top of the Cloud Storage web UI, note the path to the object. To compose the URI, replace gs://[BUCKET]/[FILE] with the appropriate path, for example, gs://mybucket/myfile.json. [BUCKET] is the Cloud Storage bucket name and [FILE] is the name of the object (file) containing the data.
You took several paragraphs to answer a question that could have been answered in one and it seems like you simply copied and pasted the answer from elsewhere
– chrisfs
Jan 22 '18 at 2:17
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f25373467%2fhow-do-i-identify-the-google-cloud-storage-uri-from-my-google-developers-console%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
6 Answers
6
active
oldest
votes
6 Answers
6
active
oldest
votes
active
oldest
votes
active
oldest
votes
The path should be gs://<bucket_name>/<file_path_inside_bucket>
.
15
Ridiculous that I had to find this answer on SO rather than in Google's own docs. Thank you @nmore!
– Stan James
Sep 14 '16 at 17:25
It's regrettable that this question did not get favorited
– chrisfs
Jan 22 '18 at 2:18
add a comment |
The path should be gs://<bucket_name>/<file_path_inside_bucket>
.
15
Ridiculous that I had to find this answer on SO rather than in Google's own docs. Thank you @nmore!
– Stan James
Sep 14 '16 at 17:25
It's regrettable that this question did not get favorited
– chrisfs
Jan 22 '18 at 2:18
add a comment |
The path should be gs://<bucket_name>/<file_path_inside_bucket>
.
The path should be gs://<bucket_name>/<file_path_inside_bucket>
.
answered Aug 18 '14 at 23:17
nmorenmore
1,472917
1,472917
15
Ridiculous that I had to find this answer on SO rather than in Google's own docs. Thank you @nmore!
– Stan James
Sep 14 '16 at 17:25
It's regrettable that this question did not get favorited
– chrisfs
Jan 22 '18 at 2:18
add a comment |
15
Ridiculous that I had to find this answer on SO rather than in Google's own docs. Thank you @nmore!
– Stan James
Sep 14 '16 at 17:25
It's regrettable that this question did not get favorited
– chrisfs
Jan 22 '18 at 2:18
15
15
Ridiculous that I had to find this answer on SO rather than in Google's own docs. Thank you @nmore!
– Stan James
Sep 14 '16 at 17:25
Ridiculous that I had to find this answer on SO rather than in Google's own docs. Thank you @nmore!
– Stan James
Sep 14 '16 at 17:25
It's regrettable that this question did not get favorited
– chrisfs
Jan 22 '18 at 2:18
It's regrettable that this question did not get favorited
– chrisfs
Jan 22 '18 at 2:18
add a comment |
Follow the following steps :
1. Go to GCS
2. Go into the folder in which the file has been uploaded
3. On the top you can see overview option
4. You can see there will be Link URL and link for GSUtil
add a comment |
Follow the following steps :
1. Go to GCS
2. Go into the folder in which the file has been uploaded
3. On the top you can see overview option
4. You can see there will be Link URL and link for GSUtil
add a comment |
Follow the following steps :
1. Go to GCS
2. Go into the folder in which the file has been uploaded
3. On the top you can see overview option
4. You can see there will be Link URL and link for GSUtil
Follow the following steps :
1. Go to GCS
2. Go into the folder in which the file has been uploaded
3. On the top you can see overview option
4. You can see there will be Link URL and link for GSUtil
answered Nov 21 '18 at 10:19


Ashish KumarAshish Kumar
17213
17213
add a comment |
add a comment |
To answer this question more information is needed. Did you already load your data into GCS?
If not, the easiest would be to go to the project console, click on project, and Storage -> Cloud Storage -> Storage browser.
You can create buckets there and upload files to the bucket.
Then the files will be found at gs://<bucket_name>/<file_path_inside_bucket>
as @nmore says.
add a comment |
To answer this question more information is needed. Did you already load your data into GCS?
If not, the easiest would be to go to the project console, click on project, and Storage -> Cloud Storage -> Storage browser.
You can create buckets there and upload files to the bucket.
Then the files will be found at gs://<bucket_name>/<file_path_inside_bucket>
as @nmore says.
add a comment |
To answer this question more information is needed. Did you already load your data into GCS?
If not, the easiest would be to go to the project console, click on project, and Storage -> Cloud Storage -> Storage browser.
You can create buckets there and upload files to the bucket.
Then the files will be found at gs://<bucket_name>/<file_path_inside_bucket>
as @nmore says.
To answer this question more information is needed. Did you already load your data into GCS?
If not, the easiest would be to go to the project console, click on project, and Storage -> Cloud Storage -> Storage browser.
You can create buckets there and upload files to the bucket.
Then the files will be found at gs://<bucket_name>/<file_path_inside_bucket>
as @nmore says.
edited Aug 23 '14 at 3:03


Misha Brukman
7,68923554
7,68923554
answered Aug 19 '14 at 4:31


Felipe HoffaFelipe Hoffa
21.5k251109
21.5k251109
add a comment |
add a comment |
Couldn't find a direct way to get the url. But found an indirect way and below are the steps:
- Go to GCS
- Go into the folder in which the file has been uploaded
- Click on the three dots at the right end of your file's row
- Click rename
- Click on gsutil equivalent link
- Copy the url alone
add a comment |
Couldn't find a direct way to get the url. But found an indirect way and below are the steps:
- Go to GCS
- Go into the folder in which the file has been uploaded
- Click on the three dots at the right end of your file's row
- Click rename
- Click on gsutil equivalent link
- Copy the url alone
add a comment |
Couldn't find a direct way to get the url. But found an indirect way and below are the steps:
- Go to GCS
- Go into the folder in which the file has been uploaded
- Click on the three dots at the right end of your file's row
- Click rename
- Click on gsutil equivalent link
- Copy the url alone
Couldn't find a direct way to get the url. But found an indirect way and below are the steps:
- Go to GCS
- Go into the folder in which the file has been uploaded
- Click on the three dots at the right end of your file's row
- Click rename
- Click on gsutil equivalent link
- Copy the url alone
answered Nov 13 '18 at 11:04
shobanashobana
12
12
add a comment |
add a comment |
If you need help on subdirectories, check this out on https://cloud.google.com/storage/docs/gsutil/addlhelp/HowSubdirectoriesWork
And https://cloud.google.com/storage/images/gsutil-subdirectories-thumb.png, if you need to see how gsutil provides a hierarchical view of objects in a bucket.
add a comment |
If you need help on subdirectories, check this out on https://cloud.google.com/storage/docs/gsutil/addlhelp/HowSubdirectoriesWork
And https://cloud.google.com/storage/images/gsutil-subdirectories-thumb.png, if you need to see how gsutil provides a hierarchical view of objects in a bucket.
add a comment |
If you need help on subdirectories, check this out on https://cloud.google.com/storage/docs/gsutil/addlhelp/HowSubdirectoriesWork
And https://cloud.google.com/storage/images/gsutil-subdirectories-thumb.png, if you need to see how gsutil provides a hierarchical view of objects in a bucket.
If you need help on subdirectories, check this out on https://cloud.google.com/storage/docs/gsutil/addlhelp/HowSubdirectoriesWork
And https://cloud.google.com/storage/images/gsutil-subdirectories-thumb.png, if you need to see how gsutil provides a hierarchical view of objects in a bucket.
answered May 16 '15 at 8:28


Derek ChiaDerek Chia
158311
158311
add a comment |
add a comment |
Retrieving the Google Cloud Storage URI
To create an external table using a Google Cloud Storage data source, you must provide the Cloud Storage URI.
The Cloud Storage URI comprises your bucket name and your object (filename). For example, if the Cloud Storage bucket is named mybucket and the data file is named myfile.csv, the bucket URI would be gs://mybucket/myfile.csv. If your data is separated into multiple files you can use a wildcard in the URI. For more information, see Cloud Storage Request URIs.
BigQuery does not support source URIs that include multiple consecutive slashes after the initial double slash. Cloud Storage object names can contain multiple consecutive slash ("/") characters. However, BigQuery converts multiple consecutives slashes into a single slash. For example, the following source URI, though valid in Cloud Storage, does not work in BigQuery: gs://[BUCKET]/my//object//name.
To retrieve the Cloud Storage URI:
Open the Cloud Storage web UI.
CLOUD STORAGE WEB UI
Browse to the location of the object (file) that contains the source data.
At the top of the Cloud Storage web UI, note the path to the object. To compose the URI, replace gs://[BUCKET]/[FILE] with the appropriate path, for example, gs://mybucket/myfile.json. [BUCKET] is the Cloud Storage bucket name and [FILE] is the name of the object (file) containing the data.
You took several paragraphs to answer a question that could have been answered in one and it seems like you simply copied and pasted the answer from elsewhere
– chrisfs
Jan 22 '18 at 2:17
add a comment |
Retrieving the Google Cloud Storage URI
To create an external table using a Google Cloud Storage data source, you must provide the Cloud Storage URI.
The Cloud Storage URI comprises your bucket name and your object (filename). For example, if the Cloud Storage bucket is named mybucket and the data file is named myfile.csv, the bucket URI would be gs://mybucket/myfile.csv. If your data is separated into multiple files you can use a wildcard in the URI. For more information, see Cloud Storage Request URIs.
BigQuery does not support source URIs that include multiple consecutive slashes after the initial double slash. Cloud Storage object names can contain multiple consecutive slash ("/") characters. However, BigQuery converts multiple consecutives slashes into a single slash. For example, the following source URI, though valid in Cloud Storage, does not work in BigQuery: gs://[BUCKET]/my//object//name.
To retrieve the Cloud Storage URI:
Open the Cloud Storage web UI.
CLOUD STORAGE WEB UI
Browse to the location of the object (file) that contains the source data.
At the top of the Cloud Storage web UI, note the path to the object. To compose the URI, replace gs://[BUCKET]/[FILE] with the appropriate path, for example, gs://mybucket/myfile.json. [BUCKET] is the Cloud Storage bucket name and [FILE] is the name of the object (file) containing the data.
You took several paragraphs to answer a question that could have been answered in one and it seems like you simply copied and pasted the answer from elsewhere
– chrisfs
Jan 22 '18 at 2:17
add a comment |
Retrieving the Google Cloud Storage URI
To create an external table using a Google Cloud Storage data source, you must provide the Cloud Storage URI.
The Cloud Storage URI comprises your bucket name and your object (filename). For example, if the Cloud Storage bucket is named mybucket and the data file is named myfile.csv, the bucket URI would be gs://mybucket/myfile.csv. If your data is separated into multiple files you can use a wildcard in the URI. For more information, see Cloud Storage Request URIs.
BigQuery does not support source URIs that include multiple consecutive slashes after the initial double slash. Cloud Storage object names can contain multiple consecutive slash ("/") characters. However, BigQuery converts multiple consecutives slashes into a single slash. For example, the following source URI, though valid in Cloud Storage, does not work in BigQuery: gs://[BUCKET]/my//object//name.
To retrieve the Cloud Storage URI:
Open the Cloud Storage web UI.
CLOUD STORAGE WEB UI
Browse to the location of the object (file) that contains the source data.
At the top of the Cloud Storage web UI, note the path to the object. To compose the URI, replace gs://[BUCKET]/[FILE] with the appropriate path, for example, gs://mybucket/myfile.json. [BUCKET] is the Cloud Storage bucket name and [FILE] is the name of the object (file) containing the data.
Retrieving the Google Cloud Storage URI
To create an external table using a Google Cloud Storage data source, you must provide the Cloud Storage URI.
The Cloud Storage URI comprises your bucket name and your object (filename). For example, if the Cloud Storage bucket is named mybucket and the data file is named myfile.csv, the bucket URI would be gs://mybucket/myfile.csv. If your data is separated into multiple files you can use a wildcard in the URI. For more information, see Cloud Storage Request URIs.
BigQuery does not support source URIs that include multiple consecutive slashes after the initial double slash. Cloud Storage object names can contain multiple consecutive slash ("/") characters. However, BigQuery converts multiple consecutives slashes into a single slash. For example, the following source URI, though valid in Cloud Storage, does not work in BigQuery: gs://[BUCKET]/my//object//name.
To retrieve the Cloud Storage URI:
Open the Cloud Storage web UI.
CLOUD STORAGE WEB UI
Browse to the location of the object (file) that contains the source data.
At the top of the Cloud Storage web UI, note the path to the object. To compose the URI, replace gs://[BUCKET]/[FILE] with the appropriate path, for example, gs://mybucket/myfile.json. [BUCKET] is the Cloud Storage bucket name and [FILE] is the name of the object (file) containing the data.
answered Jan 5 '18 at 12:29


Genesis_GDKGenesis_GDK
13
13
You took several paragraphs to answer a question that could have been answered in one and it seems like you simply copied and pasted the answer from elsewhere
– chrisfs
Jan 22 '18 at 2:17
add a comment |
You took several paragraphs to answer a question that could have been answered in one and it seems like you simply copied and pasted the answer from elsewhere
– chrisfs
Jan 22 '18 at 2:17
You took several paragraphs to answer a question that could have been answered in one and it seems like you simply copied and pasted the answer from elsewhere
– chrisfs
Jan 22 '18 at 2:17
You took several paragraphs to answer a question that could have been answered in one and it seems like you simply copied and pasted the answer from elsewhere
– chrisfs
Jan 22 '18 at 2:17
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f25373467%2fhow-do-i-identify-the-google-cloud-storage-uri-from-my-google-developers-console%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Could you post (maybe a separate question) the problems you encountered trying to set up gsutil?
– jterrace
Aug 18 '14 at 23:28
yes, how did you upload your data? If it's small enough you can upload it straight to BigQuery
– Felipe Hoffa
Aug 19 '14 at 3:05
1
I uploaded data via the google developers console. The suggestion to try gs://bucket/file name worked. This was very helpful.
– Kelly
Aug 19 '14 at 4:36