Referencing RETURN values in sequential INSERT statements, within a transaction (in Postgres)
I am attempting to insert several records across multiple tables, all in the one transaction. In this instance, its to generate some mock data. Smaller sub-sets of this insert sequence would be used in 'production'.
The records to be inserted are:
- an 'account'
- a 'gist'
- a 'version' of the gist
- several 'files', representing the actual contents of the gist version
Schema available here for reference:
https://github.com/thomasgwatson/thegistofit/tree/master/sql
My current iteration looks like this
BEGIN;
INSERT INTO account (email, name, password_hash)
VALUES
(text 'HAH@YAY.COM', text 'HAH', text 'HASHHASH')
RETURNING account_id AS new_account_id;
INSERT INTO gist (account_id)
SELECT account_id from new_account_id
RETURNING gist_id AS new_gist_id;
INSERT INTO version (gist_id, title)
SELECT gist_id, 'my gist' from new_gist_id
RETURNING version_id AS new_version_id;
INSERT INTO file (langauge, file_name, content)
VALUES
(text 'python', text 'first.py', text 'asfoasdfj'),
(text 'python', text 'second.py', text 'ipsum asdfasdf'),
(text 'python', text 'third.py', text 'okaoskdaoskd')
RETURNING file_id AS new_file_ids;
INSERT INTO version_file (version_id, file_id)
-- need to do the cartesian joint here
SELECT version_id, file_id from new_version_id, new_file_ids;
COMMIT;
I've tried a variety of ways to do this within PostgresQL, to no avail. Looking up Common Table Expressions (WITH and AS clauses) has been one main thread. However, one line from the docs might be what is sinking me:
A fine point of the above example is that the WITH clause is attached
to the INSERT, not the sub-SELECT within the INSERT. This is necessary
because data-modifying statements are only allowed in WITH clauses
that are attached to the top-level statement. However, normal WITH
visibility rules apply, so it is possible to refer to the WITH
statement's output from the sub-SELECT.
from https://www.postgresql.org/docs/current/queries-with.html
Meta commentary
- Using DataGrip as something of a linter but it's not saving me.
- SQL error messages don't seem to help much, likely that they have a slightly different mental model to error messages I'm used to
- I'd be more than comfortable using Knex/Node or Psycopg2/Python to abstract the transactions, INSERT statements, and to handle the data munging there. Trying to make it work with pure SQL
sql database postgresql
add a comment |
I am attempting to insert several records across multiple tables, all in the one transaction. In this instance, its to generate some mock data. Smaller sub-sets of this insert sequence would be used in 'production'.
The records to be inserted are:
- an 'account'
- a 'gist'
- a 'version' of the gist
- several 'files', representing the actual contents of the gist version
Schema available here for reference:
https://github.com/thomasgwatson/thegistofit/tree/master/sql
My current iteration looks like this
BEGIN;
INSERT INTO account (email, name, password_hash)
VALUES
(text 'HAH@YAY.COM', text 'HAH', text 'HASHHASH')
RETURNING account_id AS new_account_id;
INSERT INTO gist (account_id)
SELECT account_id from new_account_id
RETURNING gist_id AS new_gist_id;
INSERT INTO version (gist_id, title)
SELECT gist_id, 'my gist' from new_gist_id
RETURNING version_id AS new_version_id;
INSERT INTO file (langauge, file_name, content)
VALUES
(text 'python', text 'first.py', text 'asfoasdfj'),
(text 'python', text 'second.py', text 'ipsum asdfasdf'),
(text 'python', text 'third.py', text 'okaoskdaoskd')
RETURNING file_id AS new_file_ids;
INSERT INTO version_file (version_id, file_id)
-- need to do the cartesian joint here
SELECT version_id, file_id from new_version_id, new_file_ids;
COMMIT;
I've tried a variety of ways to do this within PostgresQL, to no avail. Looking up Common Table Expressions (WITH and AS clauses) has been one main thread. However, one line from the docs might be what is sinking me:
A fine point of the above example is that the WITH clause is attached
to the INSERT, not the sub-SELECT within the INSERT. This is necessary
because data-modifying statements are only allowed in WITH clauses
that are attached to the top-level statement. However, normal WITH
visibility rules apply, so it is possible to refer to the WITH
statement's output from the sub-SELECT.
from https://www.postgresql.org/docs/current/queries-with.html
Meta commentary
- Using DataGrip as something of a linter but it's not saving me.
- SQL error messages don't seem to help much, likely that they have a slightly different mental model to error messages I'm used to
- I'd be more than comfortable using Knex/Node or Psycopg2/Python to abstract the transactions, INSERT statements, and to handle the data munging there. Trying to make it work with pure SQL
sql database postgresql
add a comment |
I am attempting to insert several records across multiple tables, all in the one transaction. In this instance, its to generate some mock data. Smaller sub-sets of this insert sequence would be used in 'production'.
The records to be inserted are:
- an 'account'
- a 'gist'
- a 'version' of the gist
- several 'files', representing the actual contents of the gist version
Schema available here for reference:
https://github.com/thomasgwatson/thegistofit/tree/master/sql
My current iteration looks like this
BEGIN;
INSERT INTO account (email, name, password_hash)
VALUES
(text 'HAH@YAY.COM', text 'HAH', text 'HASHHASH')
RETURNING account_id AS new_account_id;
INSERT INTO gist (account_id)
SELECT account_id from new_account_id
RETURNING gist_id AS new_gist_id;
INSERT INTO version (gist_id, title)
SELECT gist_id, 'my gist' from new_gist_id
RETURNING version_id AS new_version_id;
INSERT INTO file (langauge, file_name, content)
VALUES
(text 'python', text 'first.py', text 'asfoasdfj'),
(text 'python', text 'second.py', text 'ipsum asdfasdf'),
(text 'python', text 'third.py', text 'okaoskdaoskd')
RETURNING file_id AS new_file_ids;
INSERT INTO version_file (version_id, file_id)
-- need to do the cartesian joint here
SELECT version_id, file_id from new_version_id, new_file_ids;
COMMIT;
I've tried a variety of ways to do this within PostgresQL, to no avail. Looking up Common Table Expressions (WITH and AS clauses) has been one main thread. However, one line from the docs might be what is sinking me:
A fine point of the above example is that the WITH clause is attached
to the INSERT, not the sub-SELECT within the INSERT. This is necessary
because data-modifying statements are only allowed in WITH clauses
that are attached to the top-level statement. However, normal WITH
visibility rules apply, so it is possible to refer to the WITH
statement's output from the sub-SELECT.
from https://www.postgresql.org/docs/current/queries-with.html
Meta commentary
- Using DataGrip as something of a linter but it's not saving me.
- SQL error messages don't seem to help much, likely that they have a slightly different mental model to error messages I'm used to
- I'd be more than comfortable using Knex/Node or Psycopg2/Python to abstract the transactions, INSERT statements, and to handle the data munging there. Trying to make it work with pure SQL
sql database postgresql
I am attempting to insert several records across multiple tables, all in the one transaction. In this instance, its to generate some mock data. Smaller sub-sets of this insert sequence would be used in 'production'.
The records to be inserted are:
- an 'account'
- a 'gist'
- a 'version' of the gist
- several 'files', representing the actual contents of the gist version
Schema available here for reference:
https://github.com/thomasgwatson/thegistofit/tree/master/sql
My current iteration looks like this
BEGIN;
INSERT INTO account (email, name, password_hash)
VALUES
(text 'HAH@YAY.COM', text 'HAH', text 'HASHHASH')
RETURNING account_id AS new_account_id;
INSERT INTO gist (account_id)
SELECT account_id from new_account_id
RETURNING gist_id AS new_gist_id;
INSERT INTO version (gist_id, title)
SELECT gist_id, 'my gist' from new_gist_id
RETURNING version_id AS new_version_id;
INSERT INTO file (langauge, file_name, content)
VALUES
(text 'python', text 'first.py', text 'asfoasdfj'),
(text 'python', text 'second.py', text 'ipsum asdfasdf'),
(text 'python', text 'third.py', text 'okaoskdaoskd')
RETURNING file_id AS new_file_ids;
INSERT INTO version_file (version_id, file_id)
-- need to do the cartesian joint here
SELECT version_id, file_id from new_version_id, new_file_ids;
COMMIT;
I've tried a variety of ways to do this within PostgresQL, to no avail. Looking up Common Table Expressions (WITH and AS clauses) has been one main thread. However, one line from the docs might be what is sinking me:
A fine point of the above example is that the WITH clause is attached
to the INSERT, not the sub-SELECT within the INSERT. This is necessary
because data-modifying statements are only allowed in WITH clauses
that are attached to the top-level statement. However, normal WITH
visibility rules apply, so it is possible to refer to the WITH
statement's output from the sub-SELECT.
from https://www.postgresql.org/docs/current/queries-with.html
Meta commentary
- Using DataGrip as something of a linter but it's not saving me.
- SQL error messages don't seem to help much, likely that they have a slightly different mental model to error messages I'm used to
- I'd be more than comfortable using Knex/Node or Psycopg2/Python to abstract the transactions, INSERT statements, and to handle the data munging there. Trying to make it work with pure SQL
sql database postgresql
sql database postgresql
edited Nov 20 '18 at 1:43
Underwater_developer
asked Nov 20 '18 at 1:31


Underwater_developerUnderwater_developer
176110
176110
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
Fortunately, I came up with a SQL statement that runs!
BEGIN;
with new_account as (
INSERT INTO account (email, name, password_hash)
VALUES
(text 'HAH@YAY.COM', text 'HAH', text 'HASHHASH')
RETURNING account_id
), new_gist as (
INSERT INTO gist (account_id)
SELECT account_id from new_account
RETURNING gist_id
), new_version as (
INSERT INTO version (gist_id, title)
SELECT gist_id, 'my gist' from new_gist
RETURNING version_id
), new_files as (
INSERT INTO file (langauge, file_name, content)
VALUES
(text 'python', text 'first.py', text 'ipsum hhb'),
(text 'python', text 'second.py', text 'ipsum iug'),
(text 'python', text 'third.py', text 'ipsum thd')
RETURNING file_id
)
INSERT INTO version_file (version_id, file_id)
-- yes, doing a cartesian joint here
SELECT version_id, file_id from new_version, new_files;
COMMIT;
1
I'm not precisely sure what I changed
... then why are you offering this as an answer to the community?
– Tim Biegeleisen
Nov 20 '18 at 1:32
Honestly, I was documenting my process in the question and as I was about to submit, I used the (slightly simplified) query I was about to put into the post, into DataGrip to confirm the error message and... it worked. At a broader level, I have looked multiple times across the internet, via Google, docs, Duck Duck Go, various Stack Exchanges and could not find an example of what I was trying to create. By posting it here, I hope I can save someone the hours I spent in frustration to get to this point
– Underwater_developer
Nov 20 '18 at 1:39
2
The move to CTEs is the solution. You need to consume the values returned by thereturning
clause somehow. With plain SQL you can only store them in a temporary table or pass them on by chaining CTEs
– a_horse_with_no_name
Nov 20 '18 at 6:46
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53384992%2freferencing-return-values-in-sequential-insert-statements-within-a-transaction%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Fortunately, I came up with a SQL statement that runs!
BEGIN;
with new_account as (
INSERT INTO account (email, name, password_hash)
VALUES
(text 'HAH@YAY.COM', text 'HAH', text 'HASHHASH')
RETURNING account_id
), new_gist as (
INSERT INTO gist (account_id)
SELECT account_id from new_account
RETURNING gist_id
), new_version as (
INSERT INTO version (gist_id, title)
SELECT gist_id, 'my gist' from new_gist
RETURNING version_id
), new_files as (
INSERT INTO file (langauge, file_name, content)
VALUES
(text 'python', text 'first.py', text 'ipsum hhb'),
(text 'python', text 'second.py', text 'ipsum iug'),
(text 'python', text 'third.py', text 'ipsum thd')
RETURNING file_id
)
INSERT INTO version_file (version_id, file_id)
-- yes, doing a cartesian joint here
SELECT version_id, file_id from new_version, new_files;
COMMIT;
1
I'm not precisely sure what I changed
... then why are you offering this as an answer to the community?
– Tim Biegeleisen
Nov 20 '18 at 1:32
Honestly, I was documenting my process in the question and as I was about to submit, I used the (slightly simplified) query I was about to put into the post, into DataGrip to confirm the error message and... it worked. At a broader level, I have looked multiple times across the internet, via Google, docs, Duck Duck Go, various Stack Exchanges and could not find an example of what I was trying to create. By posting it here, I hope I can save someone the hours I spent in frustration to get to this point
– Underwater_developer
Nov 20 '18 at 1:39
2
The move to CTEs is the solution. You need to consume the values returned by thereturning
clause somehow. With plain SQL you can only store them in a temporary table or pass them on by chaining CTEs
– a_horse_with_no_name
Nov 20 '18 at 6:46
add a comment |
Fortunately, I came up with a SQL statement that runs!
BEGIN;
with new_account as (
INSERT INTO account (email, name, password_hash)
VALUES
(text 'HAH@YAY.COM', text 'HAH', text 'HASHHASH')
RETURNING account_id
), new_gist as (
INSERT INTO gist (account_id)
SELECT account_id from new_account
RETURNING gist_id
), new_version as (
INSERT INTO version (gist_id, title)
SELECT gist_id, 'my gist' from new_gist
RETURNING version_id
), new_files as (
INSERT INTO file (langauge, file_name, content)
VALUES
(text 'python', text 'first.py', text 'ipsum hhb'),
(text 'python', text 'second.py', text 'ipsum iug'),
(text 'python', text 'third.py', text 'ipsum thd')
RETURNING file_id
)
INSERT INTO version_file (version_id, file_id)
-- yes, doing a cartesian joint here
SELECT version_id, file_id from new_version, new_files;
COMMIT;
1
I'm not precisely sure what I changed
... then why are you offering this as an answer to the community?
– Tim Biegeleisen
Nov 20 '18 at 1:32
Honestly, I was documenting my process in the question and as I was about to submit, I used the (slightly simplified) query I was about to put into the post, into DataGrip to confirm the error message and... it worked. At a broader level, I have looked multiple times across the internet, via Google, docs, Duck Duck Go, various Stack Exchanges and could not find an example of what I was trying to create. By posting it here, I hope I can save someone the hours I spent in frustration to get to this point
– Underwater_developer
Nov 20 '18 at 1:39
2
The move to CTEs is the solution. You need to consume the values returned by thereturning
clause somehow. With plain SQL you can only store them in a temporary table or pass them on by chaining CTEs
– a_horse_with_no_name
Nov 20 '18 at 6:46
add a comment |
Fortunately, I came up with a SQL statement that runs!
BEGIN;
with new_account as (
INSERT INTO account (email, name, password_hash)
VALUES
(text 'HAH@YAY.COM', text 'HAH', text 'HASHHASH')
RETURNING account_id
), new_gist as (
INSERT INTO gist (account_id)
SELECT account_id from new_account
RETURNING gist_id
), new_version as (
INSERT INTO version (gist_id, title)
SELECT gist_id, 'my gist' from new_gist
RETURNING version_id
), new_files as (
INSERT INTO file (langauge, file_name, content)
VALUES
(text 'python', text 'first.py', text 'ipsum hhb'),
(text 'python', text 'second.py', text 'ipsum iug'),
(text 'python', text 'third.py', text 'ipsum thd')
RETURNING file_id
)
INSERT INTO version_file (version_id, file_id)
-- yes, doing a cartesian joint here
SELECT version_id, file_id from new_version, new_files;
COMMIT;
Fortunately, I came up with a SQL statement that runs!
BEGIN;
with new_account as (
INSERT INTO account (email, name, password_hash)
VALUES
(text 'HAH@YAY.COM', text 'HAH', text 'HASHHASH')
RETURNING account_id
), new_gist as (
INSERT INTO gist (account_id)
SELECT account_id from new_account
RETURNING gist_id
), new_version as (
INSERT INTO version (gist_id, title)
SELECT gist_id, 'my gist' from new_gist
RETURNING version_id
), new_files as (
INSERT INTO file (langauge, file_name, content)
VALUES
(text 'python', text 'first.py', text 'ipsum hhb'),
(text 'python', text 'second.py', text 'ipsum iug'),
(text 'python', text 'third.py', text 'ipsum thd')
RETURNING file_id
)
INSERT INTO version_file (version_id, file_id)
-- yes, doing a cartesian joint here
SELECT version_id, file_id from new_version, new_files;
COMMIT;
edited Nov 20 '18 at 23:33
answered Nov 20 '18 at 1:31


Underwater_developerUnderwater_developer
176110
176110
1
I'm not precisely sure what I changed
... then why are you offering this as an answer to the community?
– Tim Biegeleisen
Nov 20 '18 at 1:32
Honestly, I was documenting my process in the question and as I was about to submit, I used the (slightly simplified) query I was about to put into the post, into DataGrip to confirm the error message and... it worked. At a broader level, I have looked multiple times across the internet, via Google, docs, Duck Duck Go, various Stack Exchanges and could not find an example of what I was trying to create. By posting it here, I hope I can save someone the hours I spent in frustration to get to this point
– Underwater_developer
Nov 20 '18 at 1:39
2
The move to CTEs is the solution. You need to consume the values returned by thereturning
clause somehow. With plain SQL you can only store them in a temporary table or pass them on by chaining CTEs
– a_horse_with_no_name
Nov 20 '18 at 6:46
add a comment |
1
I'm not precisely sure what I changed
... then why are you offering this as an answer to the community?
– Tim Biegeleisen
Nov 20 '18 at 1:32
Honestly, I was documenting my process in the question and as I was about to submit, I used the (slightly simplified) query I was about to put into the post, into DataGrip to confirm the error message and... it worked. At a broader level, I have looked multiple times across the internet, via Google, docs, Duck Duck Go, various Stack Exchanges and could not find an example of what I was trying to create. By posting it here, I hope I can save someone the hours I spent in frustration to get to this point
– Underwater_developer
Nov 20 '18 at 1:39
2
The move to CTEs is the solution. You need to consume the values returned by thereturning
clause somehow. With plain SQL you can only store them in a temporary table or pass them on by chaining CTEs
– a_horse_with_no_name
Nov 20 '18 at 6:46
1
1
I'm not precisely sure what I changed
... then why are you offering this as an answer to the community?– Tim Biegeleisen
Nov 20 '18 at 1:32
I'm not precisely sure what I changed
... then why are you offering this as an answer to the community?– Tim Biegeleisen
Nov 20 '18 at 1:32
Honestly, I was documenting my process in the question and as I was about to submit, I used the (slightly simplified) query I was about to put into the post, into DataGrip to confirm the error message and... it worked. At a broader level, I have looked multiple times across the internet, via Google, docs, Duck Duck Go, various Stack Exchanges and could not find an example of what I was trying to create. By posting it here, I hope I can save someone the hours I spent in frustration to get to this point
– Underwater_developer
Nov 20 '18 at 1:39
Honestly, I was documenting my process in the question and as I was about to submit, I used the (slightly simplified) query I was about to put into the post, into DataGrip to confirm the error message and... it worked. At a broader level, I have looked multiple times across the internet, via Google, docs, Duck Duck Go, various Stack Exchanges and could not find an example of what I was trying to create. By posting it here, I hope I can save someone the hours I spent in frustration to get to this point
– Underwater_developer
Nov 20 '18 at 1:39
2
2
The move to CTEs is the solution. You need to consume the values returned by the
returning
clause somehow. With plain SQL you can only store them in a temporary table or pass them on by chaining CTEs– a_horse_with_no_name
Nov 20 '18 at 6:46
The move to CTEs is the solution. You need to consume the values returned by the
returning
clause somehow. With plain SQL you can only store them in a temporary table or pass them on by chaining CTEs– a_horse_with_no_name
Nov 20 '18 at 6:46
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53384992%2freferencing-return-values-in-sequential-insert-statements-within-a-transaction%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown