Mixing
Invalid region error when using google-cloud-python API to access Dataproc
I am trying to create a cluster in Dataproc using google-cloud-python library, however, when setting region = 'us-central1' I get below exception:
google.api_core.exceptions.InvalidArgument: 400 Region 'us-central1' is invalid.
Please see https://cloud.google.com/dataproc/docs/concepts/regional-endpoints
for additional information on regional endpoints
Code (based on example):
#!/usr/bin/python
from google.cloud import dataproc_v1
client = dataproc_v1.ClusterControllerClient()
project_id = 'my-project'
region = 'us-central1'
cluster = {...}
response = client.create_cluster(project_id, region, cluster)
google-cloud-dataproc google-cloud-python
asked Nov 20 '18 at 20:44
tixtix
1,107411
add a comment |
I am trying to create a cluster in Dataproc using google-cloud-python library, however, when setting region = 'us-central1' I get below exception:
google.api_core.exceptions.InvalidArgument: 400 Region 'us-central1' is invalid.
Please see https://cloud.google.com/dataproc/docs/concepts/regional-endpoints
for additional information on regional endpoints
Code (based on example):
#!/usr/bin/python
from google.cloud import dataproc_v1
client = dataproc_v1.ClusterControllerClient()
project_id = 'my-project'
region = 'us-central1'
cluster = {...}
response = client.create_cluster(project_id, region, cluster)
google-cloud-dataproc google-cloud-python
asked Nov 20 '18 at 20:44
tixtix
1,107411
add a comment |
I am trying to create a cluster in Dataproc using google-cloud-python library, however, when setting region = 'us-central1' I get below exception:
google.api_core.exceptions.InvalidArgument: 400 Region 'us-central1' is invalid.
Please see https://cloud.google.com/dataproc/docs/concepts/regional-endpoints
for additional information on regional endpoints
Code (based on example):
#!/usr/bin/python
from google.cloud import dataproc_v1
client = dataproc_v1.ClusterControllerClient()
project_id = 'my-project'
region = 'us-central1'
cluster = {...}
response = client.create_cluster(project_id, region, cluster)
google-cloud-dataproc google-cloud-python
asked Nov 20 '18 at 20:44
tixtix
1,107411
I am trying to create a cluster in Dataproc using google-cloud-python library, however, when setting region = 'us-central1' I get below exception:
google.api_core.exceptions.InvalidArgument: 400 Region 'us-central1' is invalid.
Please see https://cloud.google.com/dataproc/docs/concepts/regional-endpoints
for additional information on regional endpoints
Code (based on example):
#!/usr/bin/python
from google.cloud import dataproc_v1
client = dataproc_v1.ClusterControllerClient()
project_id = 'my-project'
region = 'us-central1'
cluster = {...}
response = client.create_cluster(project_id, region, cluster)
google-cloud-dataproc google-cloud-python
google-cloud-dataproc google-cloud-python
asked Nov 20 '18 at 20:44
tixtix
1,107411
asked Nov 20 '18 at 20:44
tixtix
1,107411
edited Nov 20 '18 at 20:46
tix
asked Nov 20 '18 at 20:44
tixtix
1,107411
asked Nov 20 '18 at 20:44
tixtix
1,107411
asked Nov 20 '18 at 20:44
tixtix
1,107411
1,107411
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
Dataproc uses region field for routing REST requests, however, the field is not used in gRPC clients (hence the error).
Only the global multiregion can be accessed through the default endpoint. To use a regional endpoint such as us-central1, you have to configure the endpoint to address on the client's transport.
The Dataproc regional endpoints follow this pattern: <region>-dataproc.googleapis.com:443. The region field should be set to the same value as the region in the endpoint.
Example:
#!/usr/bin/python
from google.cloud import dataproc_v1
from google.cloud.dataproc_v1.gapic.transports import cluster_controller_grpc_transport
transport = cluster_controller_grpc_transport.ClusterControllerGrpcTransport(
address='us-central1-dataproc.googleapis.com:443')
client = dataproc_v1.ClusterControllerClient(transport)
project_id = 'my-project'
region = 'us-central1'
cluster = {...}
response = client.create_cluster(project_id, region, cluster)
answered Nov 20 '18 at 20:45
tixtix
1,107411
add a comment |
Similarly using the google-cloud-java client:
ClusterControllerSettings settings =
ClusterControllerSettings.newBuilder()
.setEndpoint("us-central1-dataproc.googleapis.com:443")
.build();
try (ClusterControllerClient clusterControllerClient = ClusterControllerClient.create(settings)) {
String projectId = "my-project";
String region = "us-central1";
Cluster cluster = Cluster.newBuilder().build();
Cluster response =
clusterControllerClient.createClusterAsync(projectId, region, cluster).get();
}
answered Nov 20 '18 at 20:53
tixtix
1,107411
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53401219%2finvalid-region-error-when-using-google-cloud-python-api-to-access-dataproc%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
Dataproc uses region field for routing REST requests, however, the field is not used in gRPC clients (hence the error).
Only the global multiregion can be accessed through the default endpoint. To use a regional endpoint such as us-central1, you have to configure the endpoint to address on the client's transport.
The Dataproc regional endpoints follow this pattern: <region>-dataproc.googleapis.com:443. The region field should be set to the same value as the region in the endpoint.
Example:
#!/usr/bin/python
from google.cloud import dataproc_v1
from google.cloud.dataproc_v1.gapic.transports import cluster_controller_grpc_transport
transport = cluster_controller_grpc_transport.ClusterControllerGrpcTransport(
address='us-central1-dataproc.googleapis.com:443')
client = dataproc_v1.ClusterControllerClient(transport)
project_id = 'my-project'
region = 'us-central1'
cluster = {...}
response = client.create_cluster(project_id, region, cluster)
answered Nov 20 '18 at 20:45
tixtix
1,107411
add a comment |
Dataproc uses region field for routing REST requests, however, the field is not used in gRPC clients (hence the error).
Only the global multiregion can be accessed through the default endpoint. To use a regional endpoint such as us-central1, you have to configure the endpoint to address on the client's transport.
The Dataproc regional endpoints follow this pattern: <region>-dataproc.googleapis.com:443. The region field should be set to the same value as the region in the endpoint.
Example:
#!/usr/bin/python
from google.cloud import dataproc_v1
from google.cloud.dataproc_v1.gapic.transports import cluster_controller_grpc_transport
transport = cluster_controller_grpc_transport.ClusterControllerGrpcTransport(
address='us-central1-dataproc.googleapis.com:443')
client = dataproc_v1.ClusterControllerClient(transport)
project_id = 'my-project'
region = 'us-central1'
cluster = {...}
response = client.create_cluster(project_id, region, cluster)
answered Nov 20 '18 at 20:45
tixtix
1,107411
add a comment |
Dataproc uses region field for routing REST requests, however, the field is not used in gRPC clients (hence the error).
Only the global multiregion can be accessed through the default endpoint. To use a regional endpoint such as us-central1, you have to configure the endpoint to address on the client's transport.
The Dataproc regional endpoints follow this pattern: <region>-dataproc.googleapis.com:443. The region field should be set to the same value as the region in the endpoint.
Example:
#!/usr/bin/python
from google.cloud import dataproc_v1
from google.cloud.dataproc_v1.gapic.transports import cluster_controller_grpc_transport
transport = cluster_controller_grpc_transport.ClusterControllerGrpcTransport(
address='us-central1-dataproc.googleapis.com:443')
client = dataproc_v1.ClusterControllerClient(transport)
project_id = 'my-project'
region = 'us-central1'
cluster = {...}
response = client.create_cluster(project_id, region, cluster)
answered Nov 20 '18 at 20:45
tixtix
1,107411
Dataproc uses region field for routing REST requests, however, the field is not used in gRPC clients (hence the error).
Only the global multiregion can be accessed through the default endpoint. To use a regional endpoint such as us-central1, you have to configure the endpoint to address on the client's transport.
The Dataproc regional endpoints follow this pattern: <region>-dataproc.googleapis.com:443. The region field should be set to the same value as the region in the endpoint.
Example:
#!/usr/bin/python
from google.cloud import dataproc_v1
from google.cloud.dataproc_v1.gapic.transports import cluster_controller_grpc_transport
transport = cluster_controller_grpc_transport.ClusterControllerGrpcTransport(
address='us-central1-dataproc.googleapis.com:443')
client = dataproc_v1.ClusterControllerClient(transport)
project_id = 'my-project'
region = 'us-central1'
cluster = {...}
response = client.create_cluster(project_id, region, cluster)
answered Nov 20 '18 at 20:45
tixtix
1,107411
answered Nov 20 '18 at 20:45
tixtix
1,107411
answered Nov 20 '18 at 20:45
tixtix
1,107411
answered Nov 20 '18 at 20:45
tixtix
1,107411
1,107411
add a comment |
add a comment |
Similarly using the google-cloud-java client:
ClusterControllerSettings settings =
ClusterControllerSettings.newBuilder()
.setEndpoint("us-central1-dataproc.googleapis.com:443")
.build();
try (ClusterControllerClient clusterControllerClient = ClusterControllerClient.create(settings)) {
String projectId = "my-project";
String region = "us-central1";
Cluster cluster = Cluster.newBuilder().build();
Cluster response =
clusterControllerClient.createClusterAsync(projectId, region, cluster).get();
}
answered Nov 20 '18 at 20:53
tixtix
1,107411
add a comment |
Similarly using the google-cloud-java client:
ClusterControllerSettings settings =
ClusterControllerSettings.newBuilder()
.setEndpoint("us-central1-dataproc.googleapis.com:443")
.build();
try (ClusterControllerClient clusterControllerClient = ClusterControllerClient.create(settings)) {
String projectId = "my-project";
String region = "us-central1";
Cluster cluster = Cluster.newBuilder().build();
Cluster response =
clusterControllerClient.createClusterAsync(projectId, region, cluster).get();
}
answered Nov 20 '18 at 20:53
tixtix
1,107411
add a comment |
Similarly using the google-cloud-java client:
ClusterControllerSettings settings =
ClusterControllerSettings.newBuilder()
.setEndpoint("us-central1-dataproc.googleapis.com:443")
.build();
try (ClusterControllerClient clusterControllerClient = ClusterControllerClient.create(settings)) {
String projectId = "my-project";
String region = "us-central1";
Cluster cluster = Cluster.newBuilder().build();
Cluster response =
clusterControllerClient.createClusterAsync(projectId, region, cluster).get();
}
answered Nov 20 '18 at 20:53
tixtix
1,107411
Similarly using the google-cloud-java client:
ClusterControllerSettings settings =
ClusterControllerSettings.newBuilder()
.setEndpoint("us-central1-dataproc.googleapis.com:443")
.build();
try (ClusterControllerClient clusterControllerClient = ClusterControllerClient.create(settings)) {
String projectId = "my-project";
String region = "us-central1";
Cluster cluster = Cluster.newBuilder().build();
Cluster response =
clusterControllerClient.createClusterAsync(projectId, region, cluster).get();
}
answered Nov 20 '18 at 20:53
tixtix
1,107411
answered Nov 20 '18 at 20:53
tixtix
1,107411
answered Nov 20 '18 at 20:53
tixtix
1,107411
answered Nov 20 '18 at 20:53
tixtix
1,107411
1,107411
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53401219%2finvalid-region-error-when-using-google-cloud-python-api-to-access-dataproc%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
