Mixing
Distutils causes protobuf to fail
I have a simple .proto as follows:
syntax = "proto2";
package p2p;
message dataFrame {
repeated int64 row_label = 1;
repeated int64 col_label = 2;
message data_frame_row {
repeated int32 column = 1;
}
repeated data_frame_row row = 3;
}
which I compile with protoc.
I can compile and use the following (simplified) cpp code (transitMatrix.h #includes "p2p.pb.h"):
#include "transitMatrix.h"
int main()
{
transitMatrix matrix(5, true);
matrix.addEdgeToGraph(0, 1, 5, true);
matrix.addEdgeToGraph(1, 2, 6, true);
matrix.writeTMX("outfile.tmx");
return 0;
}
However, the transitMatrix class is intended to be cythonized with distutils.
In my (simplified) test.pyx file, I cdef extern both the transitMatrix class and the dataFrame class:
cdef extern from "src/utils/serializer/p2p.pb.cc" namespace "p2p":
cdef cppclass dataFrame:
dataFrame()
cdef extern from "src/transitMatrix.h":
cdef cppclass transitMatrix:
transitMatrix()
And in my setup.py, the distutils Extension is as follows:
EXTENSION = distutils.extension.Extension(
name = 'test', language = 'c++',
sources = ['test.pyx'],
extra_compile_args = ['-Wno-unused-function',
'-std=c++11', '-Wall', '-O3', '-lprotobuf'
] + ouff_mac,
undef_macros = ["NDEBUG"],
extra_link_args = ouff_mac
)
When I run python3 setup.py install, it succeeds but warns:
clang: warning: -lprotobuf: 'linker' input unused
Finally, when I try to import the generated class in iPython (transitMatrixAdapter), I get the following error:
ImportError: dlopen(/Users/user/anaconda3/lib/python3.6/site- packages/transitMatrixAdapter.cpython-36m-darwin.so, 2): Symbol not found: __ZN6google8protobuf7Message20DiscardUnknownFieldsEv
Referenced from: /Users/user/anaconda3/lib/python3.6/site-packages/transitMatrixAdapter.cpython-36m-darwin.so
Expected in: flat namespace in /Users/user/anaconda3/lib/python3.6/site-packages/transitMatrixAdapter.cpython-36m-darwin.so
I should add that this works as expected when the protobuf dependency is removed. Why is the .so not found? I presume this is a linker error, but where is the issue?
c++ python-3.x protocol-buffers cython distutils
asked Nov 20 '18 at 19:42
lmnoellmnoel
62
add a comment |
I have a simple .proto as follows:
syntax = "proto2";
package p2p;
message dataFrame {
repeated int64 row_label = 1;
repeated int64 col_label = 2;
message data_frame_row {
repeated int32 column = 1;
}
repeated data_frame_row row = 3;
}
which I compile with protoc.
I can compile and use the following (simplified) cpp code (transitMatrix.h #includes "p2p.pb.h"):
#include "transitMatrix.h"
int main()
{
transitMatrix matrix(5, true);
matrix.addEdgeToGraph(0, 1, 5, true);
matrix.addEdgeToGraph(1, 2, 6, true);
matrix.writeTMX("outfile.tmx");
return 0;
}
However, the transitMatrix class is intended to be cythonized with distutils.
In my (simplified) test.pyx file, I cdef extern both the transitMatrix class and the dataFrame class:
cdef extern from "src/utils/serializer/p2p.pb.cc" namespace "p2p":
cdef cppclass dataFrame:
dataFrame()
cdef extern from "src/transitMatrix.h":
cdef cppclass transitMatrix:
transitMatrix()
And in my setup.py, the distutils Extension is as follows:
EXTENSION = distutils.extension.Extension(
name = 'test', language = 'c++',
sources = ['test.pyx'],
extra_compile_args = ['-Wno-unused-function',
'-std=c++11', '-Wall', '-O3', '-lprotobuf'
] + ouff_mac,
undef_macros = ["NDEBUG"],
extra_link_args = ouff_mac
)
When I run python3 setup.py install, it succeeds but warns:
clang: warning: -lprotobuf: 'linker' input unused
Finally, when I try to import the generated class in iPython (transitMatrixAdapter), I get the following error:
ImportError: dlopen(/Users/user/anaconda3/lib/python3.6/site- packages/transitMatrixAdapter.cpython-36m-darwin.so, 2): Symbol not found: __ZN6google8protobuf7Message20DiscardUnknownFieldsEv
Referenced from: /Users/user/anaconda3/lib/python3.6/site-packages/transitMatrixAdapter.cpython-36m-darwin.so
Expected in: flat namespace in /Users/user/anaconda3/lib/python3.6/site-packages/transitMatrixAdapter.cpython-36m-darwin.so
I should add that this works as expected when the protobuf dependency is removed. Why is the .so not found? I presume this is a linker error, but where is the issue?
c++ python-3.x protocol-buffers cython distutils
asked Nov 20 '18 at 19:42
lmnoellmnoel
62
-lprotobuf should be in extra link args, as clang tries to warn you
– ead
Nov 20 '18 at 20:35
Solved! Thank you very much
– lmnoel
Nov 20 '18 at 23:38
add a comment |
I have a simple .proto as follows:
syntax = "proto2";
package p2p;
message dataFrame {
repeated int64 row_label = 1;
repeated int64 col_label = 2;
message data_frame_row {
repeated int32 column = 1;
}
repeated data_frame_row row = 3;
}
which I compile with protoc.
I can compile and use the following (simplified) cpp code (transitMatrix.h #includes "p2p.pb.h"):
#include "transitMatrix.h"
int main()
{
transitMatrix matrix(5, true);
matrix.addEdgeToGraph(0, 1, 5, true);
matrix.addEdgeToGraph(1, 2, 6, true);
matrix.writeTMX("outfile.tmx");
return 0;
}
However, the transitMatrix class is intended to be cythonized with distutils.
In my (simplified) test.pyx file, I cdef extern both the transitMatrix class and the dataFrame class:
cdef extern from "src/utils/serializer/p2p.pb.cc" namespace "p2p":
cdef cppclass dataFrame:
dataFrame()
cdef extern from "src/transitMatrix.h":
cdef cppclass transitMatrix:
transitMatrix()
And in my setup.py, the distutils Extension is as follows:
EXTENSION = distutils.extension.Extension(
name = 'test', language = 'c++',
sources = ['test.pyx'],
extra_compile_args = ['-Wno-unused-function',
'-std=c++11', '-Wall', '-O3', '-lprotobuf'
] + ouff_mac,
undef_macros = ["NDEBUG"],
extra_link_args = ouff_mac
)
When I run python3 setup.py install, it succeeds but warns:
clang: warning: -lprotobuf: 'linker' input unused
Finally, when I try to import the generated class in iPython (transitMatrixAdapter), I get the following error:
ImportError: dlopen(/Users/user/anaconda3/lib/python3.6/site- packages/transitMatrixAdapter.cpython-36m-darwin.so, 2): Symbol not found: __ZN6google8protobuf7Message20DiscardUnknownFieldsEv
Referenced from: /Users/user/anaconda3/lib/python3.6/site-packages/transitMatrixAdapter.cpython-36m-darwin.so
Expected in: flat namespace in /Users/user/anaconda3/lib/python3.6/site-packages/transitMatrixAdapter.cpython-36m-darwin.so
I should add that this works as expected when the protobuf dependency is removed. Why is the .so not found? I presume this is a linker error, but where is the issue?
c++ python-3.x protocol-buffers cython distutils
asked Nov 20 '18 at 19:42
lmnoellmnoel
62
I have a simple .proto as follows:
syntax = "proto2";
package p2p;
message dataFrame {
repeated int64 row_label = 1;
repeated int64 col_label = 2;
message data_frame_row {
repeated int32 column = 1;
}
repeated data_frame_row row = 3;
}
which I compile with protoc.
I can compile and use the following (simplified) cpp code (transitMatrix.h #includes "p2p.pb.h"):
#include "transitMatrix.h"
int main()
{
transitMatrix matrix(5, true);
matrix.addEdgeToGraph(0, 1, 5, true);
matrix.addEdgeToGraph(1, 2, 6, true);
matrix.writeTMX("outfile.tmx");
return 0;
}
However, the transitMatrix class is intended to be cythonized with distutils.
In my (simplified) test.pyx file, I cdef extern both the transitMatrix class and the dataFrame class:
cdef extern from "src/utils/serializer/p2p.pb.cc" namespace "p2p":
cdef cppclass dataFrame:
dataFrame()
cdef extern from "src/transitMatrix.h":
cdef cppclass transitMatrix:
transitMatrix()
And in my setup.py, the distutils Extension is as follows:
EXTENSION = distutils.extension.Extension(
name = 'test', language = 'c++',
sources = ['test.pyx'],
extra_compile_args = ['-Wno-unused-function',
'-std=c++11', '-Wall', '-O3', '-lprotobuf'
] + ouff_mac,
undef_macros = ["NDEBUG"],
extra_link_args = ouff_mac
)
When I run python3 setup.py install, it succeeds but warns:
clang: warning: -lprotobuf: 'linker' input unused
Finally, when I try to import the generated class in iPython (transitMatrixAdapter), I get the following error:
ImportError: dlopen(/Users/user/anaconda3/lib/python3.6/site- packages/transitMatrixAdapter.cpython-36m-darwin.so, 2): Symbol not found: __ZN6google8protobuf7Message20DiscardUnknownFieldsEv
Referenced from: /Users/user/anaconda3/lib/python3.6/site-packages/transitMatrixAdapter.cpython-36m-darwin.so
Expected in: flat namespace in /Users/user/anaconda3/lib/python3.6/site-packages/transitMatrixAdapter.cpython-36m-darwin.so
I should add that this works as expected when the protobuf dependency is removed. Why is the .so not found? I presume this is a linker error, but where is the issue?
c++ python-3.x protocol-buffers cython distutils
c++ python-3.x protocol-buffers cython distutils
asked Nov 20 '18 at 19:42
lmnoellmnoel
62
asked Nov 20 '18 at 19:42
lmnoellmnoel
62
asked Nov 20 '18 at 19:42
lmnoellmnoel
62
asked Nov 20 '18 at 19:42
lmnoellmnoel
62
asked Nov 20 '18 at 19:42
lmnoellmnoel
62
62
-lprotobuf should be in extra link args, as clang tries to warn you
– ead
Nov 20 '18 at 20:35
Solved! Thank you very much
– lmnoel
Nov 20 '18 at 23:38
add a comment |
-lprotobuf should be in extra link args, as clang tries to warn you
– ead
Nov 20 '18 at 20:35
Solved! Thank you very much
– lmnoel
Nov 20 '18 at 23:38
-lprotobuf should be in extra link args, as clang tries to warn you
– ead
Nov 20 '18 at 20:35
-lprotobuf should be in extra link args, as clang tries to warn you
– ead
Nov 20 '18 at 20:35
Solved! Thank you very much
– lmnoel
Nov 20 '18 at 23:38
Solved! Thank you very much
– lmnoel
Nov 20 '18 at 23:38
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53400414%2fdistutils-causes-protobuf-to-fail%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53400414%2fdistutils-causes-protobuf-to-fail%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
-lprotobuf should be in extra link args, as clang tries to warn you
– ead
Nov 20 '18 at 20:35
Solved! Thank you very much
– lmnoel
Nov 20 '18 at 23:38