Mixing
Sphinx insert seems truncated but no errors found
0
I have a Sphinx-configuration where I can't succeed to insert some contents of documents. I have a string of almost 6MB, which I can't insert completely. I tested this by querying back the inserted value and I get back only a part of the entire content. Lets say around 0.8MB.
The configuration of Sphinx:
index RTTest
{
type = rt
path = /mnt/data001/RTTest
rt_field = Name
rt_field = Extension
rt_field = Content
rt_field = Tags
rt_attr_uint = Reference
rt_attr_uint = FileSize
rt_attr_uint = LastModified
rt_attr_uint = LastModifiedYear
rt_attr_uint = LastModifiedMonth
rt_attr_uint = LastModifiedDay
stored_fields = Content
}
searchd
{
listen = 9306:mysql41
log = /var/log/sphinxsearch/searchd.log
read_timeout = 100
max_children = 30
pid_file = /var/run/searchd.pid
max_packet_size = 128M
binlog_path = /mnt/data001
}
I have already checked the logging at /var/log/sphinxsearch/searchd.log. No errors, warnings or notices there... Neither there are errors thrown during the insert. I use the MySQL connector to connect to Sphinx as you can see in the configuration. I tested this by creating a syntax error and the error is successfully catched.
So, I have a situation where I can't insert the full content, however Sphinx nor MySQL throw or log any errors? Besides, I have been able to reproduce this issue on two other machines with an empty index using the same configuration?
What am I doing wrong?
My environment is Ubuntu 16.04LTS, Sphinx 2.2.9-id64-release and MySQL 5.7.22-0ubuntu0.16.04.1
EDIT:
Link to the file containing the insert command and the queried value
UPDATE:
I have updated to Sphinx3 and changed the rt_attr_field for docstore (stored_fields) directive. I have rebuild the index and when I'm inserting new content to the index, it still fails to insert the entire document content. I have an example below with both the query and the returned value:
INSERT INTO RTTest(id, Name, Extension, Content, Tags, Reference, FileSize, LastModified, LastModifiedYear, LastModifiedMonth, LastModifiedDay) VALUES(1774, 'DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7WtXmnUHLGt2', '7A5A5Q', 'FAUsvCA% FAUsvCA% BA8s Fg8sqy4D BA8s9jcJzE5TXysiVnBiOjr8YRE% AB0% DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7Q%% DgM% AgMjtiQ% Ag84vQ%% UFo% CwMsrTEDzA%% DgsmvCAI EQ8% CAMnrA%% AgUtqg%% CwUlvSs% BA8uuTYSx0Q% EA8woioDyQ%% EAss Eg8wvTYH AQ81vSAV1g%% CQc% Aw8s BwssrCQK Dx4ntTY% Eg8% BQInuy4DzA%% Cwsjqg%% AgM2 EQsx Cw82 Aw8s FA8mvSs% Agsjqg%% CQQ4vQ%% FQE3 Ags2uQ%% CAMnrA%% BQUwqiAF1g%% Cw8nqg%% DwQ% Dg82 FRMxrCADzw%% FR4ttiE% HAMotg%% Axg% EAUtqg%% Aw8s Fgsjqg%% CRgmvTcV Ag84vQ%% Dx4ntTY% FRMxrCALw15fbDgv CAMnrA%% AQ8xvSkDwV5Teikj DwE% HAsu Axg0tyoU HAUwvyAI Ags2 Ag84vQ%% CRo% Ag8% DwQ0tywFxw%% EAss AA8gqjAH0EM% AQ8hqiACy15Teikj AQsjrA%% EQUwvCAI HAsu DwQ% Ag8% BAMotCQBxw%% EAsx CRolvSsJz09Y EQUwvCAI AxIhrTYD0Q%% EAUtqg%% Dg82 CQQlvSgHyQ%% C0Q09iI% BA8s ABgttQ%% AgMjtiQ% AgMntCwV0U9Y AgMjtiRIxkNTczI0S2ZnH51RoVnYBIbuBLY% FQ8srA%% Eh8nqyEH2w%% AA8gqjAH0FM% V10% VFpz7Q%% VVB27Q%% Fgc% EgU% DwQ2vTcCx1lfeDU% FgYn Fg8sqy4D BQk% FAUsvCA% BA8s Fg8sqy4D FR8gsiAF1g%% AB0% DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7Q%% DQMsvA%% FA8luTcC0Q%% Cw82 EBgrvSsCx0ZfdTAi ARgtvTEDzA%% AgMjtiQ% AgMntCwV0U9Y FQ8j ABgnsSIO1g%% DwcytzcS CRonqiQSzVg% FAUqtCwB CA8mvTcKw0RS BEQ0 EQ8ntiQ% VFpy BB8rtCEPzE0% EQ8ntiQ% VV5w EgU1vTc% Bw%% V1gktA%% VVpz6g%% CAA% FAU2rCAUxktb Eg8u VVs% V1o% VFI% VF8% VFhx AAs6 VVs% V1o% VFI% VF8% VF5w AgMjtiRIxkNTczI0S2ZnH51RoVnYBIbuBLY% ER019jcJykZfeHUkV24% ABgttQ%% Eg8wvTYH FQsgtyk% Eg8wvTYHjFlXfTQreGpnK32n1;CdScZYKYkp9ov4zQ%% FQ8srA%% CwUsvCQf AA8gqjAH0FM% V1w% VFpz7Q%% UlBy4A%% Fgc% EgU% AgMjtiQ% AgMntCwV0U9Y FR8gsiAF1g%% AB0% DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7Q%% DgM% AgMjtiQ% Dw%% Dgs0vQ%% FRotsyAI EQM2sA%% DAsxtys% BwQm Dg8% Dxk% Bx0jqiA% EgIjrA%% Dw%% AgU% CAU2 Dgs0vQ%% EQ8gqywSxw%% BwkhvTYV CQQ% FAUqtCwBTZWLbA%% FQM2vQ%% Bxgn HwU3 BwguvQ%% EgU% CQg2uSwI CwU0vSgDzF4% FA8ytzcS0Q%% CQQ% Dx4ntQ%% FQ%% Dww% Dw%% CA8nvA%% EgIntQ%% BQss HwU3 FgYnuTYD Dgs0vQ%% Fg8sqy4D FhgtriwCxw%% AB8wrC0D0A%% DwQktzcLw15fcDU% AAUw EgIrqw%% AwQ2qjw% UEcouSs% FR4tuy4% BQU3tjE% Eg8wvTYH UFo% CwsrtA%% Eg8wvTYH Cws% U0dz9XdWkx8% V1947X0% Cws% U0dz9XdWkx8% V1x46XQ% EgIjti4% HwU3 Eg8wvTYH FQsgtyk% Fh8wuy0H0UNYeA%% BxkxtyYPw15T Ul5y9XdSmgcGLmx; Hll06w%% ABgttQ%% AgMjtiQ% AgMntCwV0U9Y AgMjtiRIxkNTczI0S2ZnH51RoVnYBIbuBLY% FQ8srA%% CwUsvCQf AA8gqjAH0FM% V1w% VFpz7Q%% X1By6A%% Bwc% EgU% Eg8wvTYH FQsgtyk% FR8gsiAF1g%% FA8% DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7Q%% Ag8jqg%% Eg8wvTYH FQ8n Bx42uSYOx04% DQMsvA%% FA8luTcC0Q%% Cw82 EBgrvSsCx0ZfdTAi ARgtvTEDzA%% AgMjtiQ% AgMntCwV0U9Y FQ8j ABgnsSIO1g%% DwcytzcS CRonqiQSzVg% FAUqtCwB CA8mvTcKw0RS BEQ0 EQ8ntiQ% VFpy BB8rtCEPzE0% EQ8ntiQ% VV5w EgU1vTc% Bw%% V1gktA%% VVpz6g%% CAA% FAU2rCAUxktb Eg8u VVs% V1o% VFI% VF8% VFhx AAs6 VVs% V1o% VFI% VF8% VF5w AgMjtiRIxkNTczI0S2ZnH51RoVnYBIbuBLY% ER019jcJykZfeHUkV24% ABgttQ%% Eg8wvTYH FQsgtyk% Eg8wvTYHjFlXfTQreGpnK32n1;CdScZYKYkp9ov4zQ%% FQ8srA%% CwUsvCQf AA8gqjAH0FM% V1w% VFpz7Q%% VVBy6A%% Fgc% EgU% AgMjtiQ% AgMntCwV0U9Y FR8gsiAF1g%% AB0% DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7Q%% DgM% AgMjtiQ% AgU% HwU3 Dgs0vQ%% Bw%% FQInvTE% EgIjrA%% FQItrzY% EgIn EAsx BQIjqiID0Q%% EgInoQ%% Bxgn CA8jqikf AgU3uikD EQIjrA%% EgInoQ%% Exk3uSkK2w%% Bxgn Eg8wvTYH FQsgtyk% Fh8wuy0H0UNYeA%% BxkxtyYPw15T Ul5y9XdSmgcGLmx; Hll06w%% ABgttQ%% AgMjtiQ% AgMntCwV0U9Y AgMjtiRIxkNTczI0S2ZnH51RoVnYBIbuBLY% FQ8srA%% CwUsvCQf AA8gqjAH0FM% V1w% VFpz7Q%% XlBx7w%% Bwc% EgU% Eg8wvTYH FQsgtyk% FR8gsiAF1g%% DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7Q%% Ag8jqg%% Eg8wvTYH BQss HwU3 CQE% EgIn Bx42uSYOx04% FQInvTE% EgIjti4V DQMsvA%% FA8luTcC0Q%% Cw82 EBgrvSsCx0ZfdTAi ARgtvTEDzA%% AgMjtiQ% AgMntCwV0U9Y FQ8j ABgnsSIO1g%% DwcytzcS CRonqiQSzVg% FAUqtCwB CA8mvTcKw0RS BEQ0 EQ8ntiQ% VFpy BB8rtCEPzE0% EQ8ntiQ% VV5w EgU1vTc% Bw%% V1gktA%% VVpz6g%% CAA% FAU2rCAUxktb Eg8u VVs% V1o% VFI% VF8% VFhx AAs6 VVs% V1o% VFI% VF8% VF5w AgMjtiRIxkNTczI0S2ZnH51RoVnYBIbuBLY% ER019jcJykZfeHUkV24% FgYjuyA% CQw% DB8wsTYCy0lCdjQp EgIn CA82sCAUzktYeyg% FAU2rCAUxktb EhgjvCA% FA8lsTYSx1g% FAU2rCAUxktb CBhs6nFWmhMALWs% EgIn Ah82uy0% AAUwryQUxkNYeA%% BQUsvCwSy0VYbA%% DwQ% EgIn Cgs2vTYS EA8wqywJzA%% Ag8ytzYP1k9S BBM% AA8svT0% Bx4% EgIn FA8lsTYS0FM% CQw% EgIn AgMxrDcPwV4% BQU3qjE% Bx4% FAU2rCAUxktb FQIjtCk% BxoytDw% EgIn BQUsvCwSy0VYbA%% EQMutA%% BA8% FQ8srA%% EgU% HwU3 CQQ% FA8zrSAV1g%% FhgrriwKx01Te3QkV21vNrJfYupkLoM% DwQktzcLw15fcDU% Cws7 BA8% BQUsrCQPzE9S DwQ% EgIrqw%% Cw8xqyQBxw%% BwQm FhgtrCAF1k9S BBM% Bw%% FhgtviAV0UNZcTor FhgrriwKx01T CRg% EQItqyA% AgMxuykJ0V9Eeg%% BBM% Bw%% Cgs1 Dww% HwU3 Bxgn CAU2 EgIn Bw4mqiAV0U9T DwQmsSYH1k9S DwQ% EgIrqw%% Cw8xqyQBxw%% HwU3 Cws7 CAU2 BQUyoQ%% CRg% Ag8usTMD0A%% EgIrqw%% Cw8xqyQBxw%% EgU% BwQ7tysD DwQ% FR8hsA%% BQsxvQ%% HwU3 FQItrSkC Ag8xrDcJ2w%% EgIrqw%% Cw8xqyQBxw%% BwQm CAU2sSMf Exk% DwcvvSEPw15TcyI% DwQ2vTcIx14% A0cvuSwK CA8rrC0D0A%% AR8jqiQI1k9TbA%% EgIn BQUsviwCx0RCdjorUXdw CAUw EgIn DwQ2vSIUy15P CRg% FhgtqCAU FA8hvSwW1g%% CQw% EgIn Cw8xqyQBx1k% FQ8srA%% FKn0sCkPxQ%% AgUnqw%% CAU2 BxkxrSgD BwQ7 CgMjuiwKy15P AAUw EgItqyA% BQMwuzAL0V5XcTgiSw%% Dww% EgIn Bw4mqiAV0U9T CQw% EgIrqw%% Cw8xqyQBxw%% AgUnqw%% CAU2 BQUsqyAI1g%% EgU% EgIn Exkn CQw% DwQ2vTcIx14% A0cvuSwK BwQm Cw8xqyQBxw%% FA8htzcCy0RR FgYnuTYD CAU2sSMf Exk% DwcvvSEPw15TcyI% EAsx VFpz7Q%% Ags2rSg% CRgmvTcI10dbeik% FA82rTcI Bw4vsSs% CwMs FA82rTcI DwQgtzAIxg%% BQInuy4% EQswvS0J11lT CwMs FA82rTcI Fh82uTIH2w%% EQswvS0J11lT CwMs EgA6 CgsgvSkV FQE3 DgMxrCoU2w%% FR4tuy4% BQInuy4% EwQuvTYV Fg8sqy4D BwkhtzAI1ktUcz4% AAUw FgMhsw%% Axgwtzc% AQ8wtSQI2w%% CgsgvSkV BwcjoioI CgsgvSkV CwsrtDY% BwgtrTE% EwQptioRzA%% CRgmvTcV CRgmvTc% BQIjtiID0Q%% DwQ% CRgmvTc% Dg8jvA%% Ags2uQ%% Bw4mqiAV0Q%% CRgmvTc% BQIjtiID0Q%% DwQ% Dx4ntSkPzE8% Vg%% Dx4ntQ%% CgMsvQ%% CR82 CQw% FR4tuy4% Dx4ntQ%% CgMsvQ%% BQIjtiID EgIn FR4jrDAV CQw% Bw%% FR4tuy4% Dx4ntQ%% CR4qvTcV EQM2sA%% FA8vuTcN0Q%% FA8vuTcN0Q%% EQ8nsw%% VA%% U0couStLkx8% FFhw6n1elx0E V1o% V1o% UEcouSs% FR4tuy4% BQU3tjE% Eg8wvTYH UFo% CwsrtA%% Eg8wvTYH Cws% U0dz9XdWkx8% V1947X0% Cws% U0dz9XdWkx8% V1x46XQ% UUcouSs% VFhx6HxRlRI% VVo% VVo% VA%% CwsrtA%% EAMhsyw% AgM% UEdz9XdWkx8% V1J46nA% XkcouSs% VFhw4Xxemx8% Vw%% XkcouSs% VFhx6XVXkh0% VQ%% Vw%% X0couSs% VFhx6XVXkxo% VQ%% VQ%% X0couSs% VFhx6Hxekhw% Vw%% EQ8nsw%% VQ%% V1hvsiQIjxsD VFhx6H1UkBM% VVo% VVo% V18% V1lvsiQI VFhx6XVXlRw% VQ%% Vw%% V1lvsiQI VFhx6XVXkxI% Vw%% V1lvsiQI VFhx6XVXkx8% Vw%% V1lvsiQI VFhx6XVXkxg% VA%% Vw%% V1lvsiQI VFhx6XVXkxk% Vw%% V1xvsiQI VFhx6XRUlBM% VA%% V1xvsiQI VFhx6XVemxw% VA%% V1xvsiQI VFhx6XVfkhg% Vw%% V1xvsiQI VFhx6XVemx0% Vw%% V1xvsiQI VFhx6XVUmx0% VQ%% Vw%% V1xvsiQI VFhx6XJRkx0% Uw%% Uw%% V1xvsiQI VFhx6XJQlB0% VA%% VA%% V1xvsiQI VFhx6XJSkR4% VQ%% VQ%% V1xvsiQI VFhx6XJVmhM% Vw%% Vw%% V1xvsiQI VFhx6XJVkx4% Xg%% Xg%% V1xvsiQI VFhx6XJUmxs% VA%% V1xvsiQI VFhx6XJUmxI% VA%% CgsgvSkPzE0% EgA6 CR82uioTzE4% BAU6vTY% V1xvsiQI VFhx6XJRkBw% Vw%% V1xvsiQI Ag9y6HELzw%% V18% V18% V1o% V1xvsiQI FFlz6XdXlg%% V18% UFo% V1o% V1xvsiQI Ag9y6nULzw%% V18% V18% V1o% V1xvsiQI Ag9z6nVXkx8% V18% X1o% V1o% EQ8nsw%% Vl4% V1NvsiQI VFhx6XJVlBo% Uw%% Uw%% V1NvsiQI VFhx6XJRkx8% UA%% UA%% V1NvsiQI VFhx6XJVkxg% VA%% Vw%% V1NvsiQI VFhx6XJRkB4ZLWl0CTQ6ZygcJiCK3W8vGyk% Vw%% V1NvsiQI VFhx6XJSkB0% VQ%% V1NvsiQI VFhx6XJRkR0dLWl0CTQ6b5Q% VA%% V1NvsiQI VFhx6XJVkxwdLWl0CTQ9a_I% VA%% Vw%% V1NvsiQI VFhx6XJVkxkdLWl0CTQ7Zs3zC;c6n28q2RU% Vw%% V1NvsiQI VFhx6XJVkhgdLWl0CTQ7Zp7Ntwb02r3fPhA% Vw%% V1NvsiQI VFhx6XJVkhIdLWl0CTQ6b4BTBpomHInCJM8% Vw%% V1NvsiQI VFhx6XJRkxwdLWl0CTQ6b2g% VA%% Vw%% VFpvsiQI FFt06HRXlw%% V18% UFo% V1o% VFtvsiQI FFhw63RTkRIF VFo% V18% Uw%% VFtvsiQI FA82rTcIxU9ZezI0 V1o% V1o% Uw%% VFtvsiQI FA82rTcI10EGL2oqVQ%% V18% Ul8% V18% VFhvsiQI Ulpz6HFQlxM% UVJy CgUtqyA% CgUjvCwIxQ%% BQUsrCQPzE9E UFo% CwMsqw%% CQwksSYD UVhy CwMsqw%% EQswvS0J11lT VFlvsiQI VFhx6nFWlR4% Ug%% VFlvsiQI VFhx6nZVlh4% VQ%% VFlvsiQI VFhx6nVfkRo% VQ%% VFlvsiQI VFhx6nZVlhs% VQ%% VFlvsiQI VFhx6nZVkR8% Uw%% VFlvsiQI VFhx6nBflx4% Ug%% VFlvsiQI VFhx6nNVkhI% Vw%% VFlvsiQI VFhx6nVfkxw% Ug%% VFlvsiQI VFhx6XJVkhMZLGtyFzc6bg%% Vw%% VFlvsiQI FFhw6nNTkBgC Ul8% VVo% V18% EQ8nsw%% Vl8% VFxvsiQI Ulpz6HFQlxI% UVJy CgUtqyA% CgUjvCwIxQ%% BQUsrCQPzE9E UFo% CwMsqw%% CQwksSYD UVhy CwMsqw%% EQswvS0J11lT VF1vsiQI VFhx6nNVkhw% VQ%% VQ%% VF1vsiQI VFhx6nBflRI% VA%% VA%% VF1vsiQI VFhx6nxSmxM% Vw%% Vw%% VF1vsiQI VFhx6nBflBk% VA%% Vw%% VF1vsiQI VFhx6nBfkxw% VQ%% VF1vsiQI VFhx6nBRmxw% VFhx6nNVkh4% VA%% VF1vsiQI VFhx6nBflBg% VFhx6nNWlxk% VA%% VF1vsiQI VFhx6nBflB8% Vw%% VF1vsiQI VFhx6nNVkxo% Vw%% VF1vsiQI VFhx6nNVkhk% VFhx6nBflBo% VA%% VF1vsiQI VFhx6nNVkxs% Vw%% VF1vsiQI VFhx6nBflB0% VFhx6nNWlh8% VQ%% VF1vsiQI VFhx6nBflR0% VFhx6nBekB8% VA%% VF1vsiQI FAsvuXdRkhs% V1o% V1o% V1o% VF1vsiQI EwFy6HELzw%% V18% U1o% V1o% VF1vsiQI FAsvondRkhs% V1o% V18% V1o% VFJvsiQI VFhx6nNVkh8% VQ%% VFJvsiQI VFhx6nBfkxI% VQ%% DwQhtDACy0RR EgIrqw%% CgMsvQ%% DwQ0tywFx04% CQQ% VVpt6HQ% EQ8nsw%% Vlw% VUckvSc% VFhx63dQlh0% Xg%% Xg%% VUckvSc% VFhx63dRlR0% Vw%% Vw%% VUckvSc% VFhx63dQlhs% Vw%% Vw%% VUckvSc% VFhx63dRkBM% VA%% VA%% VUckvSc% VFhx63dRlRo% Vw%% Vw%% VUckvSc% VFhx63dRkRk% Vw%% Vw%% VUckvSc% VFhx63dQkRIdLWl0CzE;a7M% VA%% VUckvSc% VFhx63dRlBIdLWl0CzE_azw% VA%% VUckvSc% VFhx63dRlxw% Vw%% VUckvSc% VFhx63FUkRo% Vw%% VUckvSc% VFhx63dRlxk% Vw%% VUckvSc% VFhx63dQlxodLWl0CzE_bkE% VA%% UkckvSc% VFhx63FUkRM% Ug%% UkckvSc% VFhx63ZelxI% Ug%% UkckvSc% VFhx63dQlh4% VQ%% UkckvSc% VFhx63dQlh8% VQ%% UkckvSc% FA82uykPwUJP V1o% V1o% V1o% UkckvSc% FA826HFWkBsD V1o% VV8% V1o% U0ckvSc% FA826ndVkxoOJmw% VFo% U1o% U1o% UEckvSc% VFhx63JSlB8% Ug%% UEckvSc% VFhx631Vlx0% Ug%% UEckvSc% VFhx631Vlx8dLWl0CzQ9ae8% Ug%% UEckvSc% VFhx631VlBo% Ug%% UEckvSc% VFhx63xXkR0% Ug%% UEckvSc% VFhx631VlxM% Ug%% UEckvSc% VFhx631VlB4% VA%% UEckvSc% VFhx631Vlxw% Ug%% UEckvSc% VFhx63xXkRw% Ug%% UEckvSc% VFhx7HRRkBk% VQ%% UEckvSc% VFhx7HRRkB0% VA%% UEckvSc% VFhx7HRRkB8% Uw%% UEckvSc% FA826ndVkxoOJmw% Ul8% Ulo% UFo% EQ8nsw%% UQ%% V1pvviAE VFhx7HRSlxw% UQ%% UQ%% V1pvviAE VFhx7HRRkxw% VA%% Uw%% V1pvviAE VFhx7HRSmh8% VA%% Uw%% V1pvviAE VFhx7HRRkBo% Vw%% Ug%% V1pvviAE VFhx7HRQkRg% Vw%% Ug%% V1pvviAE VFhx7HRSlx4% Vw%% Ug%% V1pvviAE VFhx7HRRkxM% VQ%% V1g% V1pvviAE VFhx7HRSmh4% VQ%% V1g% V1pvviAE VFhx7HRSlx0% VA%% Xg%% V1pvviAE VFhx7HRSlRI% VA%% Xg%% V1pvviAE VFhx7HVRlhg% VFhx7HRSlxk% VQ%% V1g% V1pvviAE VFhx7HRRkBs% VFhx7HRSlxg% VA%% Xg%% V1tvviAE VFhx7HVRkR0% Ug%% V1tvviAE VFhx7HVRkBk% Ug%% V1tvviAE VFhx7HVRkh8% Ug%% V1tvviAE VFhx7HVRkRI% Ug%% V1tvviAE VFhx7HVRkRM% Ug%% V1lvviAE VFhx7XRUkRgdLWl0DTI7b_FgTfqV5NfXkDQ% Ug%% V1lvviAE VFhx7XRQkB4dLWl0DTMwa8o% Ug%% V1lvviAE VFhx7HBRkhg% VA%% V1lvviAE VFhx7XRVkhs% Uw%% V1lvviAE VFhx7XdWmhw% VFhx7HJWmxo% Ug%% VFhx7XdWmxsdLWl0DDY_bc7vhv1tUNnExD4% Ug%% VFhx7XdWmx4dLWl0DDY;aptefrABxl_rE9U% Ug%% VFhx7XdWmhkdLWl0DTE5ZzASMD2MWREPqxc% Ug%% VFhx7HBRlxw% Ug%% VFhx7XRWlBk% Ug%% VFhx7HFSlB4% Ug%% VFhx7XVelh0dLWl0DTE5Zhnz;us2xm4e8uNVWaCI7x0hr7g% Ug%% VFhx7XVelx4% Ug%% VFhx7HBQkBk% Ug%% VFhx7XVelBw% Ug%% VFhx7HRRkB4% VQ%% Ag8hvSgEx1g% Vg%% Vg%% Vg%% Vg%% Vg%% Vg%% Vg%% VF0% Vg%% Vg%% Vg%% Vg%% Vg%% FA8gvSYFww%% BxoyvSkJ VA%% CwMsrTED0Q%% Fg8w CgsgvSk% FA8gvSYFww%% BxoyvSkJ VA%% CwMsrTED0Q%% Fg8w CgsgvSk% FA8gvSYFww%% BxoyvSkJ FA8gvSYFww%% BxoyvSkJ VA%% CwMsrTED0Q%% Fg8w CgsgvSk% FQInvTFX EBgrvTYL Vw%% FgsutCASzktUejc% Dxk% FR4jtiEHw1hS', '', 11, 1021089, 1503999582, 2017, 8, 29);
When I query the value with: SELECT content FROM RTTest WHERE id=1774; I get the value:
FAUsvCA% FAUsvCA% BA8s Fg8sqy4D BA8s9jcJzE5TXysiVnBiOjr8YRE% AB0% Fg8sqy4D BQUvqCkHy0RC AAUwtQ%% EgIn DgUuvCwIxQ%% BQU% DgM% Ag8jtg%% Dw%% EgItrSIO1g%% Dw%% BwYwvSQC2w%% AQs0vQ%% HwU3 BwQ% BwQxryAU CQQ% EgIrqw%% BQUvqCkHy0RC BB82 BQsstioS AAMsvA%% BwQ7 AwcjsSk% ABgttQ%% CxM% FQMmvQ%% FQU% Bx4% AAMwqzE% CxM% BxottCoBy1BTbA%% EQ8% Dgs0vQ%% DwQ0vTYSy01Xaz4j EgIrqw%% BQUvqCkHy0RC BBM% BQInuy4PzE0% CR8w FgI7qywFw0Y% FR4tuy4% BwkhtzcCy0RR EgU% CR8w Bw4vsSsP0V5Efi8uTmY% FR4tuy4% AAUwrDAIw15TcyI% EgInqyA% AgU% Cws2uy0% BwYxtw%% EgIn CB8vuiAU CQw% BAU6vTY% EQIruy0% FQItrSkC Dgs0vQ%% BA8ntg%% FgMhsyAC AAUw EgIrqw%% FQIrqCgDzF5F AgUnqw%% Cws2uy0% EgIn CB8vuiAU CQw% BAU6vTY% EQ8% FQItrSkC Dgs0vQ%% BA8ntg%% FgMhsywIxQ%% BwkhtzcCy0RR EgU% EgIrqw%% DwQktzcLw15fcDU% Dw%% BQss CQQuoQ%% BQUsuykTxk8% EgIjrA%% EQ8% Dgs0vQ%% FQIrqDUDxg%% CR82 EgIn BQUwqiAF1g%% Fx8jtjEP1kNTbA%% AAUw EgIrqw%% FQIrqCgDzF4% FQU% Fg8sqy4D CAU2 BwkhtzAI1ktUcz4% DgUyvQ%% EgU% DwQktzcLx04% HwU3 FR8kviwFy09Yaw%% FA8luTcC0Q%% BA8s ABgttQ%% BwQsuQ%% AxMu BwQsuWsD20Z2djUzXXFtOsW2NhQOEHrd7KpL FQ8srA%% CwUsvCQf Cwswuy0% VVs% VFpz7A%% V1t46XA% Fgc% EgU% Ag8jtmsLzUVEehs1V2tlNsvV0Fr0 FAUqtCwB FgYn Fg8sqy4D FR8gsiAF1g%% Fg8sqy4D BQUvqCkHy0RC AAUwtQ%% EgIn DgUuvCwIxQ%% BQU% Ag8jqg%% Ag8jtg%% Dw%% Ehg3qzE% EgIrqw%% AwcjsSk% AAMsvDY% HwU3 EQ8utA%% FgYnuTYD AAMsvA%% Bx42uSYOx04% Fg8sqy4D BQUvqCkHy0RC AAUwtQ%% ABgttQ%% EgIn DgUuvCwIxQ%% BQU% AAUw Bw%% FQItqjEHxU8% BA8xrA%% FA8luTcC0Q%% BwQsuQ%% AxMu DwQ2vTcIw15fcDUmVA%% BR8xrCoLx1g% FQ8wriwFxw%% Eg8u Xlpy UV11 UlN77Q%% AxI2 V1x3
c++ mysql sphinx
asked Nov 18 '18 at 15:10
Thomas van HesterenThomas van Hesteren
5318
add a comment |
0
I have a Sphinx-configuration where I can't succeed to insert some contents of documents. I have a string of almost 6MB, which I can't insert completely. I tested this by querying back the inserted value and I get back only a part of the entire content. Lets say around 0.8MB.
The configuration of Sphinx:
index RTTest
{
type = rt
path = /mnt/data001/RTTest
rt_field = Name
rt_field = Extension
rt_field = Content
rt_field = Tags
rt_attr_uint = Reference
rt_attr_uint = FileSize
rt_attr_uint = LastModified
rt_attr_uint = LastModifiedYear
rt_attr_uint = LastModifiedMonth
rt_attr_uint = LastModifiedDay
stored_fields = Content
}
searchd
{
listen = 9306:mysql41
log = /var/log/sphinxsearch/searchd.log
read_timeout = 100
max_children = 30
pid_file = /var/run/searchd.pid
max_packet_size = 128M
binlog_path = /mnt/data001
}
I have already checked the logging at /var/log/sphinxsearch/searchd.log. No errors, warnings or notices there... Neither there are errors thrown during the insert. I use the MySQL connector to connect to Sphinx as you can see in the configuration. I tested this by creating a syntax error and the error is successfully catched.
So, I have a situation where I can't insert the full content, however Sphinx nor MySQL throw or log any errors? Besides, I have been able to reproduce this issue on two other machines with an empty index using the same configuration?
What am I doing wrong?
My environment is Ubuntu 16.04LTS, Sphinx 2.2.9-id64-release and MySQL 5.7.22-0ubuntu0.16.04.1
EDIT:
Link to the file containing the insert command and the queried value
UPDATE:
I have updated to Sphinx3 and changed the rt_attr_field for docstore (stored_fields) directive. I have rebuild the index and when I'm inserting new content to the index, it still fails to insert the entire document content. I have an example below with both the query and the returned value:
INSERT INTO RTTest(id, Name, Extension, Content, Tags, Reference, FileSize, LastModified, LastModifiedYear, LastModifiedMonth, LastModifiedDay) VALUES(1774, 'DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7WtXmnUHLGt2', '7A5A5Q', 'FAUsvCA% FAUsvCA% BA8s Fg8sqy4D BA8s9jcJzE5TXysiVnBiOjr8YRE% AB0% DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7Q%% DgM% AgMjtiQ% Ag84vQ%% UFo% CwMsrTEDzA%% DgsmvCAI EQ8% CAMnrA%% AgUtqg%% CwUlvSs% BA8uuTYSx0Q% EA8woioDyQ%% EAss Eg8wvTYH AQ81vSAV1g%% CQc% Aw8s BwssrCQK Dx4ntTY% Eg8% BQInuy4DzA%% Cwsjqg%% AgM2 EQsx Cw82 Aw8s FA8mvSs% Agsjqg%% CQQ4vQ%% FQE3 Ags2uQ%% CAMnrA%% BQUwqiAF1g%% Cw8nqg%% DwQ% Dg82 FRMxrCADzw%% FR4ttiE% HAMotg%% Axg% EAUtqg%% Aw8s Fgsjqg%% CRgmvTcV Ag84vQ%% Dx4ntTY% FRMxrCALw15fbDgv CAMnrA%% AQ8xvSkDwV5Teikj DwE% HAsu Axg0tyoU HAUwvyAI Ags2 Ag84vQ%% CRo% Ag8% DwQ0tywFxw%% EAss AA8gqjAH0EM% AQ8hqiACy15Teikj AQsjrA%% EQUwvCAI HAsu DwQ% Ag8% BAMotCQBxw%% EAsx CRolvSsJz09Y EQUwvCAI AxIhrTYD0Q%% EAUtqg%% Dg82 CQQlvSgHyQ%% C0Q09iI% BA8s ABgttQ%% AgMjtiQ% AgMntCwV0U9Y AgMjtiRIxkNTczI0S2ZnH51RoVnYBIbuBLY% FQ8srA%% Eh8nqyEH2w%% AA8gqjAH0FM% V10% VFpz7Q%% VVB27Q%% Fgc% EgU% DwQ2vTcCx1lfeDU% FgYn Fg8sqy4D BQk% FAUsvCA% BA8s Fg8sqy4D FR8gsiAF1g%% AB0% DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7Q%% DQMsvA%% FA8luTcC0Q%% Cw82 EBgrvSsCx0ZfdTAi ARgtvTEDzA%% AgMjtiQ% AgMntCwV0U9Y FQ8j ABgnsSIO1g%% DwcytzcS CRonqiQSzVg% FAUqtCwB CA8mvTcKw0RS BEQ0 EQ8ntiQ% VFpy BB8rtCEPzE0% EQ8ntiQ% VV5w EgU1vTc% Bw%% V1gktA%% VVpz6g%% CAA% FAU2rCAUxktb Eg8u VVs% V1o% VFI% VF8% VFhx AAs6 VVs% V1o% VFI% VF8% VF5w AgMjtiRIxkNTczI0S2ZnH51RoVnYBIbuBLY% ER019jcJykZfeHUkV24% ABgttQ%% Eg8wvTYH FQsgtyk% Eg8wvTYHjFlXfTQreGpnK32n1;CdScZYKYkp9ov4zQ%% FQ8srA%% CwUsvCQf AA8gqjAH0FM% V1w% VFpz7Q%% UlBy4A%% Fgc% EgU% AgMjtiQ% AgMntCwV0U9Y FR8gsiAF1g%% AB0% DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7Q%% DgM% AgMjtiQ% Dw%% Dgs0vQ%% FRotsyAI EQM2sA%% DAsxtys% BwQm Dg8% Dxk% Bx0jqiA% EgIjrA%% Dw%% AgU% CAU2 Dgs0vQ%% EQ8gqywSxw%% BwkhvTYV CQQ% FAUqtCwBTZWLbA%% FQM2vQ%% Bxgn HwU3 BwguvQ%% EgU% CQg2uSwI CwU0vSgDzF4% FA8ytzcS0Q%% CQQ% Dx4ntQ%% FQ%% Dww% Dw%% CA8nvA%% EgIntQ%% BQss HwU3 FgYnuTYD Dgs0vQ%% Fg8sqy4D FhgtriwCxw%% AB8wrC0D0A%% DwQktzcLw15fcDU% AAUw EgIrqw%% AwQ2qjw% UEcouSs% FR4tuy4% BQU3tjE% Eg8wvTYH UFo% CwsrtA%% Eg8wvTYH Cws% U0dz9XdWkx8% V1947X0% Cws% U0dz9XdWkx8% V1x46XQ% EgIjti4% HwU3 Eg8wvTYH FQsgtyk% Fh8wuy0H0UNYeA%% BxkxtyYPw15T Ul5y9XdSmgcGLmx; Hll06w%% ABgttQ%% AgMjtiQ% AgMntCwV0U9Y AgMjtiRIxkNTczI0S2ZnH51RoVnYBIbuBLY% FQ8srA%% CwUsvCQf AA8gqjAH0FM% V1w% VFpz7Q%% X1By6A%% Bwc% EgU% Eg8wvTYH FQsgtyk% FR8gsiAF1g%% FA8% DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7Q%% Ag8jqg%% Eg8wvTYH FQ8n Bx42uSYOx04% DQMsvA%% FA8luTcC0Q%% Cw82 EBgrvSsCx0ZfdTAi ARgtvTEDzA%% AgMjtiQ% AgMntCwV0U9Y FQ8j ABgnsSIO1g%% DwcytzcS CRonqiQSzVg% FAUqtCwB CA8mvTcKw0RS BEQ0 EQ8ntiQ% VFpy BB8rtCEPzE0% EQ8ntiQ% VV5w EgU1vTc% Bw%% V1gktA%% VVpz6g%% CAA% FAU2rCAUxktb Eg8u VVs% V1o% VFI% VF8% VFhx AAs6 VVs% V1o% VFI% VF8% VF5w AgMjtiRIxkNTczI0S2ZnH51RoVnYBIbuBLY% ER019jcJykZfeHUkV24% ABgttQ%% Eg8wvTYH FQsgtyk% Eg8wvTYHjFlXfTQreGpnK32n1;CdScZYKYkp9ov4zQ%% FQ8srA%% CwUsvCQf AA8gqjAH0FM% V1w% VFpz7Q%% VVBy6A%% Fgc% EgU% AgMjtiQ% AgMntCwV0U9Y FR8gsiAF1g%% AB0% DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7Q%% DgM% AgMjtiQ% AgU% HwU3 Dgs0vQ%% Bw%% FQInvTE% EgIjrA%% FQItrzY% EgIn EAsx BQIjqiID0Q%% EgInoQ%% Bxgn CA8jqikf AgU3uikD EQIjrA%% EgInoQ%% Exk3uSkK2w%% Bxgn Eg8wvTYH FQsgtyk% Fh8wuy0H0UNYeA%% BxkxtyYPw15T Ul5y9XdSmgcGLmx; Hll06w%% ABgttQ%% AgMjtiQ% AgMntCwV0U9Y AgMjtiRIxkNTczI0S2ZnH51RoVnYBIbuBLY% FQ8srA%% CwUsvCQf AA8gqjAH0FM% V1w% VFpz7Q%% XlBx7w%% Bwc% EgU% Eg8wvTYH FQsgtyk% FR8gsiAF1g%% DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7Q%% Ag8jqg%% Eg8wvTYH BQss HwU3 CQE% EgIn Bx42uSYOx04% FQInvTE% EgIjti4V DQMsvA%% FA8luTcC0Q%% Cw82 EBgrvSsCx0ZfdTAi ARgtvTEDzA%% AgMjtiQ% AgMntCwV0U9Y FQ8j ABgnsSIO1g%% DwcytzcS CRonqiQSzVg% FAUqtCwB CA8mvTcKw0RS BEQ0 EQ8ntiQ% VFpy BB8rtCEPzE0% EQ8ntiQ% VV5w EgU1vTc% Bw%% V1gktA%% VVpz6g%% CAA% FAU2rCAUxktb Eg8u VVs% V1o% VFI% VF8% VFhx AAs6 VVs% V1o% VFI% VF8% VF5w AgMjtiRIxkNTczI0S2ZnH51RoVnYBIbuBLY% ER019jcJykZfeHUkV24% FgYjuyA% CQw% DB8wsTYCy0lCdjQp EgIn CA82sCAUzktYeyg% FAU2rCAUxktb EhgjvCA% FA8lsTYSx1g% FAU2rCAUxktb CBhs6nFWmhMALWs% EgIn Ah82uy0% AAUwryQUxkNYeA%% BQUsvCwSy0VYbA%% DwQ% EgIn Cgs2vTYS EA8wqywJzA%% Ag8ytzYP1k9S BBM% AA8svT0% Bx4% EgIn FA8lsTYS0FM% CQw% EgIn AgMxrDcPwV4% BQU3qjE% Bx4% FAU2rCAUxktb FQIjtCk% BxoytDw% EgIn BQUsvCwSy0VYbA%% EQMutA%% BA8% FQ8srA%% EgU% HwU3 CQQ% FA8zrSAV1g%% FhgrriwKx01Te3QkV21vNrJfYupkLoM% DwQktzcLw15fcDU% Cws7 BA8% BQUsrCQPzE9S DwQ% EgIrqw%% Cw8xqyQBxw%% BwQm FhgtrCAF1k9S BBM% Bw%% FhgtviAV0UNZcTor FhgrriwKx01T CRg% EQItqyA% AgMxuykJ0V9Eeg%% BBM% Bw%% Cgs1 Dww% HwU3 Bxgn CAU2 EgIn Bw4mqiAV0U9T DwQmsSYH1k9S DwQ% EgIrqw%% Cw8xqyQBxw%% HwU3 Cws7 CAU2 BQUyoQ%% CRg% Ag8usTMD0A%% EgIrqw%% Cw8xqyQBxw%% EgU% BwQ7tysD DwQ% FR8hsA%% BQsxvQ%% HwU3 FQItrSkC Ag8xrDcJ2w%% EgIrqw%% Cw8xqyQBxw%% BwQm CAU2sSMf Exk% DwcvvSEPw15TcyI% DwQ2vTcIx14% A0cvuSwK CA8rrC0D0A%% AR8jqiQI1k9TbA%% EgIn BQUsviwCx0RCdjorUXdw CAUw EgIn DwQ2vSIUy15P CRg% FhgtqCAU FA8hvSwW1g%% CQw% EgIn Cw8xqyQBx1k% FQ8srA%% FKn0sCkPxQ%% AgUnqw%% CAU2 BxkxrSgD BwQ7 CgMjuiwKy15P AAUw EgItqyA% BQMwuzAL0V5XcTgiSw%% Dww% EgIn Bw4mqiAV0U9T CQw% EgIrqw%% Cw8xqyQBxw%% AgUnqw%% CAU2 BQUsqyAI1g%% EgU% EgIn Exkn CQw% DwQ2vTcIx14% A0cvuSwK BwQm Cw8xqyQBxw%% FA8htzcCy0RR FgYnuTYD CAU2sSMf Exk% DwcvvSEPw15TcyI% EAsx VFpz7Q%% Ags2rSg% CRgmvTcI10dbeik% FA82rTcI Bw4vsSs% CwMs FA82rTcI DwQgtzAIxg%% BQInuy4% EQswvS0J11lT CwMs FA82rTcI Fh82uTIH2w%% EQswvS0J11lT CwMs EgA6 CgsgvSkV FQE3 DgMxrCoU2w%% FR4tuy4% BQInuy4% EwQuvTYV Fg8sqy4D BwkhtzAI1ktUcz4% AAUw FgMhsw%% Axgwtzc% AQ8wtSQI2w%% CgsgvSkV BwcjoioI CgsgvSkV CwsrtDY% BwgtrTE% EwQptioRzA%% CRgmvTcV CRgmvTc% BQIjtiID0Q%% DwQ% CRgmvTc% Dg8jvA%% Ags2uQ%% Bw4mqiAV0Q%% CRgmvTc% BQIjtiID0Q%% DwQ% Dx4ntSkPzE8% Vg%% Dx4ntQ%% CgMsvQ%% CR82 CQw% FR4tuy4% Dx4ntQ%% CgMsvQ%% BQIjtiID EgIn FR4jrDAV CQw% Bw%% FR4tuy4% Dx4ntQ%% CR4qvTcV EQM2sA%% FA8vuTcN0Q%% FA8vuTcN0Q%% EQ8nsw%% VA%% U0couStLkx8% FFhw6n1elx0E V1o% V1o% UEcouSs% FR4tuy4% BQU3tjE% Eg8wvTYH UFo% CwsrtA%% Eg8wvTYH Cws% U0dz9XdWkx8% V1947X0% Cws% U0dz9XdWkx8% V1x46XQ% UUcouSs% VFhx6HxRlRI% VVo% VVo% VA%% CwsrtA%% EAMhsyw% AgM% UEdz9XdWkx8% V1J46nA% XkcouSs% VFhw4Xxemx8% Vw%% XkcouSs% VFhx6XVXkh0% VQ%% Vw%% X0couSs% VFhx6XVXkxo% VQ%% VQ%% X0couSs% VFhx6Hxekhw% Vw%% EQ8nsw%% VQ%% V1hvsiQIjxsD VFhx6H1UkBM% VVo% VVo% V18% V1lvsiQI VFhx6XVXlRw% VQ%% Vw%% V1lvsiQI VFhx6XVXkxI% Vw%% V1lvsiQI VFhx6XVXkx8% Vw%% V1lvsiQI VFhx6XVXkxg% VA%% Vw%% V1lvsiQI VFhx6XVXkxk% Vw%% V1xvsiQI VFhx6XRUlBM% VA%% V1xvsiQI VFhx6XVemxw% VA%% V1xvsiQI VFhx6XVfkhg% Vw%% V1xvsiQI VFhx6XVemx0% Vw%% V1xvsiQI VFhx6XVUmx0% VQ%% Vw%% V1xvsiQI VFhx6XJRkx0% Uw%% Uw%% V1xvsiQI VFhx6XJQlB0% VA%% VA%% V1xvsiQI VFhx6XJSkR4% VQ%% VQ%% V1xvsiQI VFhx6XJVmhM% Vw%% Vw%% V1xvsiQI VFhx6XJVkx4% Xg%% Xg%% V1xvsiQI VFhx6XJUmxs% VA%% V1xvsiQI VFhx6XJUmxI% VA%% CgsgvSkPzE0% EgA6 CR82uioTzE4% BAU6vTY% V1xvsiQI VFhx6XJRkBw% Vw%% V1xvsiQI Ag9y6HELzw%% V18% V18% V1o% V1xvsiQI FFlz6XdXlg%% V18% UFo% V1o% V1xvsiQI Ag9y6nULzw%% V18% V18% V1o% V1xvsiQI Ag9z6nVXkx8% V18% X1o% V1o% EQ8nsw%% Vl4% V1NvsiQI VFhx6XJVlBo% Uw%% Uw%% V1NvsiQI VFhx6XJRkx8% UA%% UA%% V1NvsiQI VFhx6XJVkxg% VA%% Vw%% V1NvsiQI VFhx6XJRkB4ZLWl0CTQ6ZygcJiCK3W8vGyk% Vw%% V1NvsiQI VFhx6XJSkB0% VQ%% V1NvsiQI VFhx6XJRkR0dLWl0CTQ6b5Q% VA%% V1NvsiQI VFhx6XJVkxwdLWl0CTQ9a_I% VA%% Vw%% V1NvsiQI VFhx6XJVkxkdLWl0CTQ7Zs3zC;c6n28q2RU% Vw%% V1NvsiQI VFhx6XJVkhgdLWl0CTQ7Zp7Ntwb02r3fPhA% Vw%% V1NvsiQI VFhx6XJVkhIdLWl0CTQ6b4BTBpomHInCJM8% Vw%% V1NvsiQI VFhx6XJRkxwdLWl0CTQ6b2g% VA%% Vw%% VFpvsiQI FFt06HRXlw%% V18% UFo% V1o% VFtvsiQI FFhw63RTkRIF VFo% V18% Uw%% VFtvsiQI FA82rTcIxU9ZezI0 V1o% V1o% Uw%% VFtvsiQI FA82rTcI10EGL2oqVQ%% V18% Ul8% V18% VFhvsiQI Ulpz6HFQlxM% UVJy CgUtqyA% CgUjvCwIxQ%% BQUsrCQPzE9E UFo% CwMsqw%% CQwksSYD UVhy CwMsqw%% EQswvS0J11lT VFlvsiQI VFhx6nFWlR4% Ug%% VFlvsiQI VFhx6nZVlh4% VQ%% VFlvsiQI VFhx6nVfkRo% VQ%% VFlvsiQI VFhx6nZVlhs% VQ%% VFlvsiQI VFhx6nZVkR8% Uw%% VFlvsiQI VFhx6nBflx4% Ug%% VFlvsiQI VFhx6nNVkhI% Vw%% VFlvsiQI VFhx6nVfkxw% Ug%% VFlvsiQI VFhx6XJVkhMZLGtyFzc6bg%% Vw%% VFlvsiQI FFhw6nNTkBgC Ul8% VVo% V18% EQ8nsw%% Vl8% VFxvsiQI Ulpz6HFQlxI% UVJy CgUtqyA% CgUjvCwIxQ%% BQUsrCQPzE9E UFo% CwMsqw%% CQwksSYD UVhy CwMsqw%% EQswvS0J11lT VF1vsiQI VFhx6nNVkhw% VQ%% VQ%% VF1vsiQI VFhx6nBflRI% VA%% VA%% VF1vsiQI VFhx6nxSmxM% Vw%% Vw%% VF1vsiQI VFhx6nBflBk% VA%% Vw%% VF1vsiQI VFhx6nBfkxw% VQ%% VF1vsiQI VFhx6nBRmxw% VFhx6nNVkh4% VA%% VF1vsiQI VFhx6nBflBg% VFhx6nNWlxk% VA%% VF1vsiQI VFhx6nBflB8% Vw%% VF1vsiQI VFhx6nNVkxo% Vw%% VF1vsiQI VFhx6nNVkhk% VFhx6nBflBo% VA%% VF1vsiQI VFhx6nNVkxs% Vw%% VF1vsiQI VFhx6nBflB0% VFhx6nNWlh8% VQ%% VF1vsiQI VFhx6nBflR0% VFhx6nBekB8% VA%% VF1vsiQI FAsvuXdRkhs% V1o% V1o% V1o% VF1vsiQI EwFy6HELzw%% V18% U1o% V1o% VF1vsiQI FAsvondRkhs% V1o% V18% V1o% VFJvsiQI VFhx6nNVkh8% VQ%% VFJvsiQI VFhx6nBfkxI% VQ%% DwQhtDACy0RR EgIrqw%% CgMsvQ%% DwQ0tywFx04% CQQ% VVpt6HQ% EQ8nsw%% Vlw% VUckvSc% VFhx63dQlh0% Xg%% Xg%% VUckvSc% VFhx63dRlR0% Vw%% Vw%% VUckvSc% VFhx63dQlhs% Vw%% Vw%% VUckvSc% VFhx63dRkBM% VA%% VA%% VUckvSc% VFhx63dRlRo% Vw%% Vw%% VUckvSc% VFhx63dRkRk% Vw%% Vw%% VUckvSc% VFhx63dQkRIdLWl0CzE;a7M% VA%% VUckvSc% VFhx63dRlBIdLWl0CzE_azw% VA%% VUckvSc% VFhx63dRlxw% Vw%% VUckvSc% VFhx63FUkRo% Vw%% VUckvSc% VFhx63dRlxk% Vw%% VUckvSc% VFhx63dQlxodLWl0CzE_bkE% VA%% UkckvSc% VFhx63FUkRM% Ug%% UkckvSc% VFhx63ZelxI% Ug%% UkckvSc% VFhx63dQlh4% VQ%% UkckvSc% VFhx63dQlh8% VQ%% UkckvSc% FA82uykPwUJP V1o% V1o% V1o% UkckvSc% FA826HFWkBsD V1o% VV8% V1o% U0ckvSc% FA826ndVkxoOJmw% VFo% U1o% U1o% UEckvSc% VFhx63JSlB8% Ug%% UEckvSc% VFhx631Vlx0% Ug%% UEckvSc% VFhx631Vlx8dLWl0CzQ9ae8% Ug%% UEckvSc% VFhx631VlBo% Ug%% UEckvSc% VFhx63xXkR0% Ug%% UEckvSc% VFhx631VlxM% Ug%% UEckvSc% VFhx631VlB4% VA%% UEckvSc% VFhx631Vlxw% Ug%% UEckvSc% VFhx63xXkRw% Ug%% UEckvSc% VFhx7HRRkBk% VQ%% UEckvSc% VFhx7HRRkB0% VA%% UEckvSc% VFhx7HRRkB8% Uw%% UEckvSc% FA826ndVkxoOJmw% Ul8% Ulo% UFo% EQ8nsw%% UQ%% V1pvviAE VFhx7HRSlxw% UQ%% UQ%% V1pvviAE VFhx7HRRkxw% VA%% Uw%% V1pvviAE VFhx7HRSmh8% VA%% Uw%% V1pvviAE VFhx7HRRkBo% Vw%% Ug%% V1pvviAE VFhx7HRQkRg% Vw%% Ug%% V1pvviAE VFhx7HRSlx4% Vw%% Ug%% V1pvviAE VFhx7HRRkxM% VQ%% V1g% V1pvviAE VFhx7HRSmh4% VQ%% V1g% V1pvviAE VFhx7HRSlx0% VA%% Xg%% V1pvviAE VFhx7HRSlRI% VA%% Xg%% V1pvviAE VFhx7HVRlhg% VFhx7HRSlxk% VQ%% V1g% V1pvviAE VFhx7HRRkBs% VFhx7HRSlxg% VA%% Xg%% V1tvviAE VFhx7HVRkR0% Ug%% V1tvviAE VFhx7HVRkBk% Ug%% V1tvviAE VFhx7HVRkh8% Ug%% V1tvviAE VFhx7HVRkRI% Ug%% V1tvviAE VFhx7HVRkRM% Ug%% V1lvviAE VFhx7XRUkRgdLWl0DTI7b_FgTfqV5NfXkDQ% Ug%% V1lvviAE VFhx7XRQkB4dLWl0DTMwa8o% Ug%% V1lvviAE VFhx7HBRkhg% VA%% V1lvviAE VFhx7XRVkhs% Uw%% V1lvviAE VFhx7XdWmhw% VFhx7HJWmxo% Ug%% VFhx7XdWmxsdLWl0DDY_bc7vhv1tUNnExD4% Ug%% VFhx7XdWmx4dLWl0DDY;aptefrABxl_rE9U% Ug%% VFhx7XdWmhkdLWl0DTE5ZzASMD2MWREPqxc% Ug%% VFhx7HBRlxw% Ug%% VFhx7XRWlBk% Ug%% VFhx7HFSlB4% Ug%% VFhx7XVelh0dLWl0DTE5Zhnz;us2xm4e8uNVWaCI7x0hr7g% Ug%% VFhx7XVelx4% Ug%% VFhx7HBQkBk% Ug%% VFhx7XVelBw% Ug%% VFhx7HRRkB4% VQ%% Ag8hvSgEx1g% Vg%% Vg%% Vg%% Vg%% Vg%% Vg%% Vg%% VF0% Vg%% Vg%% Vg%% Vg%% Vg%% FA8gvSYFww%% BxoyvSkJ VA%% CwMsrTED0Q%% Fg8w CgsgvSk% FA8gvSYFww%% BxoyvSkJ VA%% CwMsrTED0Q%% Fg8w CgsgvSk% FA8gvSYFww%% BxoyvSkJ FA8gvSYFww%% BxoyvSkJ VA%% CwMsrTED0Q%% Fg8w CgsgvSk% FQInvTFX EBgrvTYL Vw%% FgsutCASzktUejc% Dxk% FR4jtiEHw1hS', '', 11, 1021089, 1503999582, 2017, 8, 29);
When I query the value with: SELECT content FROM RTTest WHERE id=1774; I get the value:
FAUsvCA% FAUsvCA% BA8s Fg8sqy4D BA8s9jcJzE5TXysiVnBiOjr8YRE% AB0% Fg8sqy4D BQUvqCkHy0RC AAUwtQ%% EgIn DgUuvCwIxQ%% BQU% DgM% Ag8jtg%% Dw%% EgItrSIO1g%% Dw%% BwYwvSQC2w%% AQs0vQ%% HwU3 BwQ% BwQxryAU CQQ% EgIrqw%% BQUvqCkHy0RC BB82 BQsstioS AAMsvA%% BwQ7 AwcjsSk% ABgttQ%% CxM% FQMmvQ%% FQU% Bx4% AAMwqzE% CxM% BxottCoBy1BTbA%% EQ8% Dgs0vQ%% DwQ0vTYSy01Xaz4j EgIrqw%% BQUvqCkHy0RC BBM% BQInuy4PzE0% CR8w FgI7qywFw0Y% FR4tuy4% BwkhtzcCy0RR EgU% CR8w Bw4vsSsP0V5Efi8uTmY% FR4tuy4% AAUwrDAIw15TcyI% EgInqyA% AgU% Cws2uy0% BwYxtw%% EgIn CB8vuiAU CQw% BAU6vTY% EQIruy0% FQItrSkC Dgs0vQ%% BA8ntg%% FgMhsyAC AAUw EgIrqw%% FQIrqCgDzF5F AgUnqw%% Cws2uy0% EgIn CB8vuiAU CQw% BAU6vTY% EQ8% FQItrSkC Dgs0vQ%% BA8ntg%% FgMhsywIxQ%% BwkhtzcCy0RR EgU% EgIrqw%% DwQktzcLw15fcDU% Dw%% BQss CQQuoQ%% BQUsuykTxk8% EgIjrA%% EQ8% Dgs0vQ%% FQIrqDUDxg%% CR82 EgIn BQUwqiAF1g%% Fx8jtjEP1kNTbA%% AAUw EgIrqw%% FQIrqCgDzF4% FQU% Fg8sqy4D CAU2 BwkhtzAI1ktUcz4% DgUyvQ%% EgU% DwQktzcLx04% HwU3 FR8kviwFy09Yaw%% FA8luTcC0Q%% BA8s ABgttQ%% BwQsuQ%% AxMu BwQsuWsD20Z2djUzXXFtOsW2NhQOEHrd7KpL FQ8srA%% CwUsvCQf Cwswuy0% VVs% VFpz7A%% V1t46XA% Fgc% EgU% Ag8jtmsLzUVEehs1V2tlNsvV0Fr0 FAUqtCwB FgYn Fg8sqy4D FR8gsiAF1g%% Fg8sqy4D BQUvqCkHy0RC AAUwtQ%% EgIn DgUuvCwIxQ%% BQU% Ag8jqg%% Ag8jtg%% Dw%% Ehg3qzE% EgIrqw%% AwcjsSk% AAMsvDY% HwU3 EQ8utA%% FgYnuTYD AAMsvA%% Bx42uSYOx04% Fg8sqy4D BQUvqCkHy0RC AAUwtQ%% ABgttQ%% EgIn DgUuvCwIxQ%% BQU% AAUw Bw%% FQItqjEHxU8% BA8xrA%% FA8luTcC0Q%% BwQsuQ%% AxMu DwQ2vTcIw15fcDUmVA%% BR8xrCoLx1g% FQ8wriwFxw%% Eg8u Xlpy UV11 UlN77Q%% AxI2 V1x3
c++ mysql sphinx
asked Nov 18 '18 at 15:10
Thomas van HesterenThomas van Hesteren
5318
Hi. Can you provide the full insert command which reproduces the problem?
– Manticore Search
Nov 18 '18 at 16:11
@Manticore-Sphinxnextgen, see the edited question. I have added a link to the insert command and the result I get back from the query. Others have already tried the command and they get no errors but don't respond anymore with the used configuration. Please let me know what configuration you use if you succeed to store the string?
– Thomas van Hesteren
Nov 18 '18 at 16:22
add a comment |
0
0
0
I have a Sphinx-configuration where I can't succeed to insert some contents of documents. I have a string of almost 6MB, which I can't insert completely. I tested this by querying back the inserted value and I get back only a part of the entire content. Lets say around 0.8MB.
The configuration of Sphinx:
index RTTest
{
type = rt
path = /mnt/data001/RTTest
rt_field = Name
rt_field = Extension
rt_field = Content
rt_field = Tags
rt_attr_uint = Reference
rt_attr_uint = FileSize
rt_attr_uint = LastModified
rt_attr_uint = LastModifiedYear
rt_attr_uint = LastModifiedMonth
rt_attr_uint = LastModifiedDay
stored_fields = Content
}
searchd
{
listen = 9306:mysql41
log = /var/log/sphinxsearch/searchd.log
read_timeout = 100
max_children = 30
pid_file = /var/run/searchd.pid
max_packet_size = 128M
binlog_path = /mnt/data001
}
I have already checked the logging at /var/log/sphinxsearch/searchd.log. No errors, warnings or notices there... Neither there are errors thrown during the insert. I use the MySQL connector to connect to Sphinx as you can see in the configuration. I tested this by creating a syntax error and the error is successfully catched.
So, I have a situation where I can't insert the full content, however Sphinx nor MySQL throw or log any errors? Besides, I have been able to reproduce this issue on two other machines with an empty index using the same configuration?
What am I doing wrong?
My environment is Ubuntu 16.04LTS, Sphinx 2.2.9-id64-release and MySQL 5.7.22-0ubuntu0.16.04.1
EDIT:
Link to the file containing the insert command and the queried value
UPDATE:
I have updated to Sphinx3 and changed the rt_attr_field for docstore (stored_fields) directive. I have rebuild the index and when I'm inserting new content to the index, it still fails to insert the entire document content. I have an example below with both the query and the returned value:
INSERT INTO RTTest(id, Name, Extension, Content, Tags, Reference, FileSize, LastModified, LastModifiedYear, LastModifiedMonth, LastModifiedDay) VALUES(1774, 'DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7WtXmnUHLGt2', '7A5A5Q', 'FAUsvCA% FAUsvCA% BA8s Fg8sqy4D BA8s9jcJzE5TXysiVnBiOjr8YRE% AB0% DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7Q%% DgM% AgMjtiQ% Ag84vQ%% UFo% CwMsrTEDzA%% DgsmvCAI EQ8% CAMnrA%% AgUtqg%% CwUlvSs% BA8uuTYSx0Q% EA8woioDyQ%% EAss Eg8wvTYH AQ81vSAV1g%% CQc% Aw8s BwssrCQK Dx4ntTY% Eg8% BQInuy4DzA%% Cwsjqg%% AgM2 EQsx Cw82 Aw8s FA8mvSs% Agsjqg%% CQQ4vQ%% FQE3 Ags2uQ%% CAMnrA%% BQUwqiAF1g%% Cw8nqg%% DwQ% Dg82 FRMxrCADzw%% FR4ttiE% HAMotg%% Axg% EAUtqg%% Aw8s Fgsjqg%% CRgmvTcV Ag84vQ%% Dx4ntTY% FRMxrCALw15fbDgv CAMnrA%% AQ8xvSkDwV5Teikj DwE% HAsu Axg0tyoU HAUwvyAI Ags2 Ag84vQ%% CRo% Ag8% DwQ0tywFxw%% EAss AA8gqjAH0EM% AQ8hqiACy15Teikj AQsjrA%% EQUwvCAI HAsu DwQ% Ag8% BAMotCQBxw%% EAsx CRolvSsJz09Y EQUwvCAI AxIhrTYD0Q%% EAUtqg%% Dg82 CQQlvSgHyQ%% C0Q09iI% BA8s ABgttQ%% AgMjtiQ% AgMntCwV0U9Y AgMjtiRIxkNTczI0S2ZnH51RoVnYBIbuBLY% FQ8srA%% Eh8nqyEH2w%% AA8gqjAH0FM% V10% VFpz7Q%% VVB27Q%% Fgc% EgU% DwQ2vTcCx1lfeDU% FgYn Fg8sqy4D BQk% FAUsvCA% BA8s Fg8sqy4D FR8gsiAF1g%% AB0% DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7Q%% DQMsvA%% FA8luTcC0Q%% Cw82 EBgrvSsCx0ZfdTAi ARgtvTEDzA%% AgMjtiQ% AgMntCwV0U9Y FQ8j ABgnsSIO1g%% DwcytzcS CRonqiQSzVg% FAUqtCwB CA8mvTcKw0RS BEQ0 EQ8ntiQ% VFpy BB8rtCEPzE0% EQ8ntiQ% VV5w EgU1vTc% Bw%% V1gktA%% VVpz6g%% CAA% FAU2rCAUxktb Eg8u VVs% V1o% VFI% VF8% VFhx AAs6 VVs% V1o% VFI% VF8% VF5w AgMjtiRIxkNTczI0S2ZnH51RoVnYBIbuBLY% ER019jcJykZfeHUkV24% ABgttQ%% Eg8wvTYH FQsgtyk% Eg8wvTYHjFlXfTQreGpnK32n1;CdScZYKYkp9ov4zQ%% FQ8srA%% CwUsvCQf AA8gqjAH0FM% V1w% VFpz7Q%% UlBy4A%% Fgc% EgU% AgMjtiQ% AgMntCwV0U9Y FR8gsiAF1g%% AB0% DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7Q%% DgM% AgMjtiQ% Dw%% Dgs0vQ%% FRotsyAI EQM2sA%% DAsxtys% BwQm Dg8% Dxk% Bx0jqiA% EgIjrA%% Dw%% AgU% CAU2 Dgs0vQ%% EQ8gqywSxw%% BwkhvTYV CQQ% FAUqtCwBTZWLbA%% FQM2vQ%% Bxgn HwU3 BwguvQ%% EgU% CQg2uSwI CwU0vSgDzF4% FA8ytzcS0Q%% CQQ% Dx4ntQ%% FQ%% Dww% Dw%% CA8nvA%% EgIntQ%% BQss HwU3 FgYnuTYD Dgs0vQ%% Fg8sqy4D FhgtriwCxw%% AB8wrC0D0A%% DwQktzcLw15fcDU% AAUw EgIrqw%% AwQ2qjw% UEcouSs% FR4tuy4% BQU3tjE% Eg8wvTYH UFo% CwsrtA%% Eg8wvTYH Cws% U0dz9XdWkx8% V1947X0% Cws% U0dz9XdWkx8% V1x46XQ% EgIjti4% HwU3 Eg8wvTYH FQsgtyk% Fh8wuy0H0UNYeA%% BxkxtyYPw15T Ul5y9XdSmgcGLmx; Hll06w%% ABgttQ%% AgMjtiQ% AgMntCwV0U9Y AgMjtiRIxkNTczI0S2ZnH51RoVnYBIbuBLY% FQ8srA%% CwUsvCQf AA8gqjAH0FM% V1w% VFpz7Q%% X1By6A%% Bwc% EgU% Eg8wvTYH FQsgtyk% FR8gsiAF1g%% FA8% DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7Q%% Ag8jqg%% Eg8wvTYH FQ8n Bx42uSYOx04% DQMsvA%% FA8luTcC0Q%% Cw82 EBgrvSsCx0ZfdTAi ARgtvTEDzA%% AgMjtiQ% AgMntCwV0U9Y FQ8j ABgnsSIO1g%% DwcytzcS CRonqiQSzVg% FAUqtCwB CA8mvTcKw0RS BEQ0 EQ8ntiQ% VFpy BB8rtCEPzE0% EQ8ntiQ% VV5w EgU1vTc% Bw%% V1gktA%% VVpz6g%% CAA% FAU2rCAUxktb Eg8u VVs% V1o% VFI% VF8% VFhx AAs6 VVs% V1o% VFI% VF8% VF5w AgMjtiRIxkNTczI0S2ZnH51RoVnYBIbuBLY% ER019jcJykZfeHUkV24% ABgttQ%% Eg8wvTYH FQsgtyk% Eg8wvTYHjFlXfTQreGpnK32n1;CdScZYKYkp9ov4zQ%% FQ8srA%% CwUsvCQf AA8gqjAH0FM% V1w% VFpz7Q%% VVBy6A%% Fgc% EgU% AgMjtiQ% AgMntCwV0U9Y FR8gsiAF1g%% AB0% DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7Q%% DgM% AgMjtiQ% AgU% HwU3 Dgs0vQ%% Bw%% FQInvTE% EgIjrA%% FQItrzY% EgIn EAsx BQIjqiID0Q%% EgInoQ%% Bxgn CA8jqikf AgU3uikD EQIjrA%% EgInoQ%% Exk3uSkK2w%% Bxgn Eg8wvTYH FQsgtyk% Fh8wuy0H0UNYeA%% BxkxtyYPw15T Ul5y9XdSmgcGLmx; Hll06w%% ABgttQ%% AgMjtiQ% AgMntCwV0U9Y AgMjtiRIxkNTczI0S2ZnH51RoVnYBIbuBLY% FQ8srA%% CwUsvCQf AA8gqjAH0FM% V1w% VFpz7Q%% XlBx7w%% Bwc% EgU% Eg8wvTYH FQsgtyk% FR8gsiAF1g%% DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7Q%% Ag8jqg%% Eg8wvTYH BQss HwU3 CQE% EgIn Bx42uSYOx04% FQInvTE% EgIjti4V DQMsvA%% FA8luTcC0Q%% Cw82 EBgrvSsCx0ZfdTAi ARgtvTEDzA%% AgMjtiQ% AgMntCwV0U9Y FQ8j ABgnsSIO1g%% DwcytzcS CRonqiQSzVg% FAUqtCwB CA8mvTcKw0RS BEQ0 EQ8ntiQ% VFpy BB8rtCEPzE0% EQ8ntiQ% VV5w EgU1vTc% Bw%% V1gktA%% VVpz6g%% CAA% FAU2rCAUxktb Eg8u VVs% V1o% VFI% VF8% VFhx AAs6 VVs% V1o% VFI% VF8% VF5w AgMjtiRIxkNTczI0S2ZnH51RoVnYBIbuBLY% ER019jcJykZfeHUkV24% FgYjuyA% CQw% DB8wsTYCy0lCdjQp EgIn CA82sCAUzktYeyg% FAU2rCAUxktb EhgjvCA% FA8lsTYSx1g% FAU2rCAUxktb CBhs6nFWmhMALWs% EgIn Ah82uy0% AAUwryQUxkNYeA%% BQUsvCwSy0VYbA%% DwQ% EgIn Cgs2vTYS EA8wqywJzA%% Ag8ytzYP1k9S BBM% AA8svT0% Bx4% EgIn FA8lsTYS0FM% CQw% EgIn AgMxrDcPwV4% BQU3qjE% Bx4% FAU2rCAUxktb FQIjtCk% BxoytDw% EgIn BQUsvCwSy0VYbA%% EQMutA%% BA8% FQ8srA%% EgU% HwU3 CQQ% FA8zrSAV1g%% FhgrriwKx01Te3QkV21vNrJfYupkLoM% DwQktzcLw15fcDU% Cws7 BA8% BQUsrCQPzE9S DwQ% EgIrqw%% Cw8xqyQBxw%% BwQm FhgtrCAF1k9S BBM% Bw%% FhgtviAV0UNZcTor FhgrriwKx01T CRg% EQItqyA% AgMxuykJ0V9Eeg%% BBM% Bw%% Cgs1 Dww% HwU3 Bxgn CAU2 EgIn Bw4mqiAV0U9T DwQmsSYH1k9S DwQ% EgIrqw%% Cw8xqyQBxw%% HwU3 Cws7 CAU2 BQUyoQ%% CRg% Ag8usTMD0A%% EgIrqw%% Cw8xqyQBxw%% EgU% BwQ7tysD DwQ% FR8hsA%% BQsxvQ%% HwU3 FQItrSkC Ag8xrDcJ2w%% EgIrqw%% Cw8xqyQBxw%% BwQm CAU2sSMf Exk% DwcvvSEPw15TcyI% DwQ2vTcIx14% A0cvuSwK CA8rrC0D0A%% AR8jqiQI1k9TbA%% EgIn BQUsviwCx0RCdjorUXdw CAUw EgIn DwQ2vSIUy15P CRg% FhgtqCAU FA8hvSwW1g%% CQw% EgIn Cw8xqyQBx1k% FQ8srA%% FKn0sCkPxQ%% AgUnqw%% CAU2 BxkxrSgD BwQ7 CgMjuiwKy15P AAUw EgItqyA% BQMwuzAL0V5XcTgiSw%% Dww% EgIn Bw4mqiAV0U9T CQw% EgIrqw%% Cw8xqyQBxw%% AgUnqw%% CAU2 BQUsqyAI1g%% EgU% EgIn Exkn CQw% DwQ2vTcIx14% A0cvuSwK BwQm Cw8xqyQBxw%% FA8htzcCy0RR FgYnuTYD CAU2sSMf Exk% DwcvvSEPw15TcyI% EAsx VFpz7Q%% Ags2rSg% CRgmvTcI10dbeik% FA82rTcI Bw4vsSs% CwMs FA82rTcI DwQgtzAIxg%% BQInuy4% EQswvS0J11lT CwMs FA82rTcI Fh82uTIH2w%% EQswvS0J11lT CwMs EgA6 CgsgvSkV FQE3 DgMxrCoU2w%% FR4tuy4% BQInuy4% EwQuvTYV Fg8sqy4D BwkhtzAI1ktUcz4% AAUw FgMhsw%% Axgwtzc% AQ8wtSQI2w%% CgsgvSkV BwcjoioI CgsgvSkV CwsrtDY% BwgtrTE% EwQptioRzA%% CRgmvTcV CRgmvTc% BQIjtiID0Q%% DwQ% CRgmvTc% Dg8jvA%% Ags2uQ%% Bw4mqiAV0Q%% CRgmvTc% BQIjtiID0Q%% DwQ% Dx4ntSkPzE8% Vg%% Dx4ntQ%% CgMsvQ%% CR82 CQw% FR4tuy4% Dx4ntQ%% CgMsvQ%% BQIjtiID EgIn FR4jrDAV CQw% Bw%% FR4tuy4% Dx4ntQ%% CR4qvTcV EQM2sA%% FA8vuTcN0Q%% FA8vuTcN0Q%% EQ8nsw%% VA%% U0couStLkx8% FFhw6n1elx0E V1o% V1o% UEcouSs% FR4tuy4% BQU3tjE% Eg8wvTYH UFo% CwsrtA%% Eg8wvTYH Cws% U0dz9XdWkx8% V1947X0% Cws% U0dz9XdWkx8% V1x46XQ% UUcouSs% VFhx6HxRlRI% VVo% VVo% VA%% CwsrtA%% EAMhsyw% AgM% UEdz9XdWkx8% V1J46nA% XkcouSs% VFhw4Xxemx8% Vw%% XkcouSs% VFhx6XVXkh0% VQ%% Vw%% X0couSs% VFhx6XVXkxo% VQ%% VQ%% X0couSs% VFhx6Hxekhw% Vw%% EQ8nsw%% VQ%% V1hvsiQIjxsD VFhx6H1UkBM% VVo% VVo% V18% V1lvsiQI VFhx6XVXlRw% VQ%% Vw%% V1lvsiQI VFhx6XVXkxI% Vw%% V1lvsiQI VFhx6XVXkx8% Vw%% V1lvsiQI VFhx6XVXkxg% VA%% Vw%% V1lvsiQI VFhx6XVXkxk% Vw%% V1xvsiQI VFhx6XRUlBM% VA%% V1xvsiQI VFhx6XVemxw% VA%% V1xvsiQI VFhx6XVfkhg% Vw%% V1xvsiQI VFhx6XVemx0% Vw%% V1xvsiQI VFhx6XVUmx0% VQ%% Vw%% V1xvsiQI VFhx6XJRkx0% Uw%% Uw%% V1xvsiQI VFhx6XJQlB0% VA%% VA%% V1xvsiQI VFhx6XJSkR4% VQ%% VQ%% V1xvsiQI VFhx6XJVmhM% Vw%% Vw%% V1xvsiQI VFhx6XJVkx4% Xg%% Xg%% V1xvsiQI VFhx6XJUmxs% VA%% V1xvsiQI VFhx6XJUmxI% VA%% CgsgvSkPzE0% EgA6 CR82uioTzE4% BAU6vTY% V1xvsiQI VFhx6XJRkBw% Vw%% V1xvsiQI Ag9y6HELzw%% V18% V18% V1o% V1xvsiQI FFlz6XdXlg%% V18% UFo% V1o% V1xvsiQI Ag9y6nULzw%% V18% V18% V1o% V1xvsiQI Ag9z6nVXkx8% V18% X1o% V1o% EQ8nsw%% Vl4% V1NvsiQI VFhx6XJVlBo% Uw%% Uw%% V1NvsiQI VFhx6XJRkx8% UA%% UA%% V1NvsiQI VFhx6XJVkxg% VA%% Vw%% V1NvsiQI VFhx6XJRkB4ZLWl0CTQ6ZygcJiCK3W8vGyk% Vw%% V1NvsiQI VFhx6XJSkB0% VQ%% V1NvsiQI VFhx6XJRkR0dLWl0CTQ6b5Q% VA%% V1NvsiQI VFhx6XJVkxwdLWl0CTQ9a_I% VA%% Vw%% V1NvsiQI VFhx6XJVkxkdLWl0CTQ7Zs3zC;c6n28q2RU% Vw%% V1NvsiQI VFhx6XJVkhgdLWl0CTQ7Zp7Ntwb02r3fPhA% Vw%% V1NvsiQI VFhx6XJVkhIdLWl0CTQ6b4BTBpomHInCJM8% Vw%% V1NvsiQI VFhx6XJRkxwdLWl0CTQ6b2g% VA%% Vw%% VFpvsiQI FFt06HRXlw%% V18% UFo% V1o% VFtvsiQI FFhw63RTkRIF VFo% V18% Uw%% VFtvsiQI FA82rTcIxU9ZezI0 V1o% V1o% Uw%% VFtvsiQI FA82rTcI10EGL2oqVQ%% V18% Ul8% V18% VFhvsiQI Ulpz6HFQlxM% UVJy CgUtqyA% CgUjvCwIxQ%% BQUsrCQPzE9E UFo% CwMsqw%% CQwksSYD UVhy CwMsqw%% EQswvS0J11lT VFlvsiQI VFhx6nFWlR4% Ug%% VFlvsiQI VFhx6nZVlh4% VQ%% VFlvsiQI VFhx6nVfkRo% VQ%% VFlvsiQI VFhx6nZVlhs% VQ%% VFlvsiQI VFhx6nZVkR8% Uw%% VFlvsiQI VFhx6nBflx4% Ug%% VFlvsiQI VFhx6nNVkhI% Vw%% VFlvsiQI VFhx6nVfkxw% Ug%% VFlvsiQI VFhx6XJVkhMZLGtyFzc6bg%% Vw%% VFlvsiQI FFhw6nNTkBgC Ul8% VVo% V18% EQ8nsw%% Vl8% VFxvsiQI Ulpz6HFQlxI% UVJy CgUtqyA% CgUjvCwIxQ%% BQUsrCQPzE9E UFo% CwMsqw%% CQwksSYD UVhy CwMsqw%% EQswvS0J11lT VF1vsiQI VFhx6nNVkhw% VQ%% VQ%% VF1vsiQI VFhx6nBflRI% VA%% VA%% VF1vsiQI VFhx6nxSmxM% Vw%% Vw%% VF1vsiQI VFhx6nBflBk% VA%% Vw%% VF1vsiQI VFhx6nBfkxw% VQ%% VF1vsiQI VFhx6nBRmxw% VFhx6nNVkh4% VA%% VF1vsiQI VFhx6nBflBg% VFhx6nNWlxk% VA%% VF1vsiQI VFhx6nBflB8% Vw%% VF1vsiQI VFhx6nNVkxo% Vw%% VF1vsiQI VFhx6nNVkhk% VFhx6nBflBo% VA%% VF1vsiQI VFhx6nNVkxs% Vw%% VF1vsiQI VFhx6nBflB0% VFhx6nNWlh8% VQ%% VF1vsiQI VFhx6nBflR0% VFhx6nBekB8% VA%% VF1vsiQI FAsvuXdRkhs% V1o% V1o% V1o% VF1vsiQI EwFy6HELzw%% V18% U1o% V1o% VF1vsiQI FAsvondRkhs% V1o% V18% V1o% VFJvsiQI VFhx6nNVkh8% VQ%% VFJvsiQI VFhx6nBfkxI% VQ%% DwQhtDACy0RR EgIrqw%% CgMsvQ%% DwQ0tywFx04% CQQ% VVpt6HQ% EQ8nsw%% Vlw% VUckvSc% VFhx63dQlh0% Xg%% Xg%% VUckvSc% VFhx63dRlR0% Vw%% Vw%% VUckvSc% VFhx63dQlhs% Vw%% Vw%% VUckvSc% VFhx63dRkBM% VA%% VA%% VUckvSc% VFhx63dRlRo% Vw%% Vw%% VUckvSc% VFhx63dRkRk% Vw%% Vw%% VUckvSc% VFhx63dQkRIdLWl0CzE;a7M% VA%% VUckvSc% VFhx63dRlBIdLWl0CzE_azw% VA%% VUckvSc% VFhx63dRlxw% Vw%% VUckvSc% VFhx63FUkRo% Vw%% VUckvSc% VFhx63dRlxk% Vw%% VUckvSc% VFhx63dQlxodLWl0CzE_bkE% VA%% UkckvSc% VFhx63FUkRM% Ug%% UkckvSc% VFhx63ZelxI% Ug%% UkckvSc% VFhx63dQlh4% VQ%% UkckvSc% VFhx63dQlh8% VQ%% UkckvSc% FA82uykPwUJP V1o% V1o% V1o% UkckvSc% FA826HFWkBsD V1o% VV8% V1o% U0ckvSc% FA826ndVkxoOJmw% VFo% U1o% U1o% UEckvSc% VFhx63JSlB8% Ug%% UEckvSc% VFhx631Vlx0% Ug%% UEckvSc% VFhx631Vlx8dLWl0CzQ9ae8% Ug%% UEckvSc% VFhx631VlBo% Ug%% UEckvSc% VFhx63xXkR0% Ug%% UEckvSc% VFhx631VlxM% Ug%% UEckvSc% VFhx631VlB4% VA%% UEckvSc% VFhx631Vlxw% Ug%% UEckvSc% VFhx63xXkRw% Ug%% UEckvSc% VFhx7HRRkBk% VQ%% UEckvSc% VFhx7HRRkB0% VA%% UEckvSc% VFhx7HRRkB8% Uw%% UEckvSc% FA826ndVkxoOJmw% Ul8% Ulo% UFo% EQ8nsw%% UQ%% V1pvviAE VFhx7HRSlxw% UQ%% UQ%% V1pvviAE VFhx7HRRkxw% VA%% Uw%% V1pvviAE VFhx7HRSmh8% VA%% Uw%% V1pvviAE VFhx7HRRkBo% Vw%% Ug%% V1pvviAE VFhx7HRQkRg% Vw%% Ug%% V1pvviAE VFhx7HRSlx4% Vw%% Ug%% V1pvviAE VFhx7HRRkxM% VQ%% V1g% V1pvviAE VFhx7HRSmh4% VQ%% V1g% V1pvviAE VFhx7HRSlx0% VA%% Xg%% V1pvviAE VFhx7HRSlRI% VA%% Xg%% V1pvviAE VFhx7HVRlhg% VFhx7HRSlxk% VQ%% V1g% V1pvviAE VFhx7HRRkBs% VFhx7HRSlxg% VA%% Xg%% V1tvviAE VFhx7HVRkR0% Ug%% V1tvviAE VFhx7HVRkBk% Ug%% V1tvviAE VFhx7HVRkh8% Ug%% V1tvviAE VFhx7HVRkRI% Ug%% V1tvviAE VFhx7HVRkRM% Ug%% V1lvviAE VFhx7XRUkRgdLWl0DTI7b_FgTfqV5NfXkDQ% Ug%% V1lvviAE VFhx7XRQkB4dLWl0DTMwa8o% Ug%% V1lvviAE VFhx7HBRkhg% VA%% V1lvviAE VFhx7XRVkhs% Uw%% V1lvviAE VFhx7XdWmhw% VFhx7HJWmxo% Ug%% VFhx7XdWmxsdLWl0DDY_bc7vhv1tUNnExD4% Ug%% VFhx7XdWmx4dLWl0DDY;aptefrABxl_rE9U% Ug%% VFhx7XdWmhkdLWl0DTE5ZzASMD2MWREPqxc% Ug%% VFhx7HBRlxw% Ug%% VFhx7XRWlBk% Ug%% VFhx7HFSlB4% Ug%% VFhx7XVelh0dLWl0DTE5Zhnz;us2xm4e8uNVWaCI7x0hr7g% Ug%% VFhx7XVelx4% Ug%% VFhx7HBQkBk% Ug%% VFhx7XVelBw% Ug%% VFhx7HRRkB4% VQ%% Ag8hvSgEx1g% Vg%% Vg%% Vg%% Vg%% Vg%% Vg%% Vg%% VF0% Vg%% Vg%% Vg%% Vg%% Vg%% FA8gvSYFww%% BxoyvSkJ VA%% CwMsrTED0Q%% Fg8w CgsgvSk% FA8gvSYFww%% BxoyvSkJ VA%% CwMsrTED0Q%% Fg8w CgsgvSk% FA8gvSYFww%% BxoyvSkJ FA8gvSYFww%% BxoyvSkJ VA%% CwMsrTED0Q%% Fg8w CgsgvSk% FQInvTFX EBgrvTYL Vw%% FgsutCASzktUejc% Dxk% FR4jtiEHw1hS', '', 11, 1021089, 1503999582, 2017, 8, 29);
When I query the value with: SELECT content FROM RTTest WHERE id=1774; I get the value:
FAUsvCA% FAUsvCA% BA8s Fg8sqy4D BA8s9jcJzE5TXysiVnBiOjr8YRE% AB0% Fg8sqy4D BQUvqCkHy0RC AAUwtQ%% EgIn DgUuvCwIxQ%% BQU% DgM% Ag8jtg%% Dw%% EgItrSIO1g%% Dw%% BwYwvSQC2w%% AQs0vQ%% HwU3 BwQ% BwQxryAU CQQ% EgIrqw%% BQUvqCkHy0RC BB82 BQsstioS AAMsvA%% BwQ7 AwcjsSk% ABgttQ%% CxM% FQMmvQ%% FQU% Bx4% AAMwqzE% CxM% BxottCoBy1BTbA%% EQ8% Dgs0vQ%% DwQ0vTYSy01Xaz4j EgIrqw%% BQUvqCkHy0RC BBM% BQInuy4PzE0% CR8w FgI7qywFw0Y% FR4tuy4% BwkhtzcCy0RR EgU% CR8w Bw4vsSsP0V5Efi8uTmY% FR4tuy4% AAUwrDAIw15TcyI% EgInqyA% AgU% Cws2uy0% BwYxtw%% EgIn CB8vuiAU CQw% BAU6vTY% EQIruy0% FQItrSkC Dgs0vQ%% BA8ntg%% FgMhsyAC AAUw EgIrqw%% FQIrqCgDzF5F AgUnqw%% Cws2uy0% EgIn CB8vuiAU CQw% BAU6vTY% EQ8% FQItrSkC Dgs0vQ%% BA8ntg%% FgMhsywIxQ%% BwkhtzcCy0RR EgU% EgIrqw%% DwQktzcLw15fcDU% Dw%% BQss CQQuoQ%% BQUsuykTxk8% EgIjrA%% EQ8% Dgs0vQ%% FQIrqDUDxg%% CR82 EgIn BQUwqiAF1g%% Fx8jtjEP1kNTbA%% AAUw EgIrqw%% FQIrqCgDzF4% FQU% Fg8sqy4D CAU2 BwkhtzAI1ktUcz4% DgUyvQ%% EgU% DwQktzcLx04% HwU3 FR8kviwFy09Yaw%% FA8luTcC0Q%% BA8s ABgttQ%% BwQsuQ%% AxMu BwQsuWsD20Z2djUzXXFtOsW2NhQOEHrd7KpL FQ8srA%% CwUsvCQf Cwswuy0% VVs% VFpz7A%% V1t46XA% Fgc% EgU% Ag8jtmsLzUVEehs1V2tlNsvV0Fr0 FAUqtCwB FgYn Fg8sqy4D FR8gsiAF1g%% Fg8sqy4D BQUvqCkHy0RC AAUwtQ%% EgIn DgUuvCwIxQ%% BQU% Ag8jqg%% Ag8jtg%% Dw%% Ehg3qzE% EgIrqw%% AwcjsSk% AAMsvDY% HwU3 EQ8utA%% FgYnuTYD AAMsvA%% Bx42uSYOx04% Fg8sqy4D BQUvqCkHy0RC AAUwtQ%% ABgttQ%% EgIn DgUuvCwIxQ%% BQU% AAUw Bw%% FQItqjEHxU8% BA8xrA%% FA8luTcC0Q%% BwQsuQ%% AxMu DwQ2vTcIw15fcDUmVA%% BR8xrCoLx1g% FQ8wriwFxw%% Eg8u Xlpy UV11 UlN77Q%% AxI2 V1x3
c++ mysql sphinx
asked Nov 18 '18 at 15:10
Thomas van HesterenThomas van Hesteren
5318
I have a Sphinx-configuration where I can't succeed to insert some contents of documents. I have a string of almost 6MB, which I can't insert completely. I tested this by querying back the inserted value and I get back only a part of the entire content. Lets say around 0.8MB.
The configuration of Sphinx:
index RTTest
{
type = rt
path = /mnt/data001/RTTest
rt_field = Name
rt_field = Extension
rt_field = Content
rt_field = Tags
rt_attr_uint = Reference
rt_attr_uint = FileSize
rt_attr_uint = LastModified
rt_attr_uint = LastModifiedYear
rt_attr_uint = LastModifiedMonth
rt_attr_uint = LastModifiedDay
stored_fields = Content
}
searchd
{
listen = 9306:mysql41
log = /var/log/sphinxsearch/searchd.log
read_timeout = 100
max_children = 30
pid_file = /var/run/searchd.pid
max_packet_size = 128M
binlog_path = /mnt/data001
}
I have already checked the logging at /var/log/sphinxsearch/searchd.log. No errors, warnings or notices there... Neither there are errors thrown during the insert. I use the MySQL connector to connect to Sphinx as you can see in the configuration. I tested this by creating a syntax error and the error is successfully catched.
So, I have a situation where I can't insert the full content, however Sphinx nor MySQL throw or log any errors? Besides, I have been able to reproduce this issue on two other machines with an empty index using the same configuration?
What am I doing wrong?
My environment is Ubuntu 16.04LTS, Sphinx 2.2.9-id64-release and MySQL 5.7.22-0ubuntu0.16.04.1
EDIT:
Link to the file containing the insert command and the queried value
UPDATE:
I have updated to Sphinx3 and changed the rt_attr_field for docstore (stored_fields) directive. I have rebuild the index and when I'm inserting new content to the index, it still fails to insert the entire document content. I have an example below with both the query and the returned value:
INSERT INTO RTTest(id, Name, Extension, Content, Tags, Reference, FileSize, LastModified, LastModifiedYear, LastModifiedMonth, LastModifiedDay) VALUES(1774, 'DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7WtXmnUHLGt2', '7A5A5Q', 'FAUsvCA% FAUsvCA% BA8s Fg8sqy4D BA8s9jcJzE5TXysiVnBiOjr8YRE% AB0% DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7Q%% DgM% AgMjtiQ% Ag84vQ%% UFo% CwMsrTEDzA%% DgsmvCAI EQ8% CAMnrA%% AgUtqg%% CwUlvSs% BA8uuTYSx0Q% EA8woioDyQ%% EAss Eg8wvTYH AQ81vSAV1g%% CQc% Aw8s BwssrCQK Dx4ntTY% Eg8% BQInuy4DzA%% Cwsjqg%% AgM2 EQsx Cw82 Aw8s FA8mvSs% Agsjqg%% CQQ4vQ%% FQE3 Ags2uQ%% CAMnrA%% BQUwqiAF1g%% Cw8nqg%% DwQ% Dg82 FRMxrCADzw%% FR4ttiE% HAMotg%% Axg% EAUtqg%% Aw8s Fgsjqg%% CRgmvTcV Ag84vQ%% Dx4ntTY% FRMxrCALw15fbDgv CAMnrA%% AQ8xvSkDwV5Teikj DwE% HAsu Axg0tyoU HAUwvyAI Ags2 Ag84vQ%% CRo% Ag8% DwQ0tywFxw%% EAss AA8gqjAH0EM% AQ8hqiACy15Teikj AQsjrA%% EQUwvCAI HAsu DwQ% Ag8% BAMotCQBxw%% EAsx CRolvSsJz09Y EQUwvCAI AxIhrTYD0Q%% EAUtqg%% Dg82 CQQlvSgHyQ%% C0Q09iI% BA8s ABgttQ%% AgMjtiQ% AgMntCwV0U9Y AgMjtiRIxkNTczI0S2ZnH51RoVnYBIbuBLY% FQ8srA%% Eh8nqyEH2w%% AA8gqjAH0FM% V10% VFpz7Q%% VVB27Q%% Fgc% EgU% DwQ2vTcCx1lfeDU% FgYn Fg8sqy4D BQk% FAUsvCA% BA8s Fg8sqy4D FR8gsiAF1g%% AB0% DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7Q%% DQMsvA%% FA8luTcC0Q%% Cw82 EBgrvSsCx0ZfdTAi ARgtvTEDzA%% AgMjtiQ% AgMntCwV0U9Y FQ8j ABgnsSIO1g%% DwcytzcS CRonqiQSzVg% FAUqtCwB CA8mvTcKw0RS BEQ0 EQ8ntiQ% VFpy BB8rtCEPzE0% EQ8ntiQ% VV5w EgU1vTc% Bw%% V1gktA%% VVpz6g%% CAA% FAU2rCAUxktb Eg8u VVs% V1o% VFI% VF8% VFhx AAs6 VVs% V1o% VFI% VF8% VF5w AgMjtiRIxkNTczI0S2ZnH51RoVnYBIbuBLY% ER019jcJykZfeHUkV24% ABgttQ%% Eg8wvTYH FQsgtyk% Eg8wvTYHjFlXfTQreGpnK32n1;CdScZYKYkp9ov4zQ%% FQ8srA%% CwUsvCQf AA8gqjAH0FM% V1w% VFpz7Q%% UlBy4A%% Fgc% EgU% AgMjtiQ% AgMntCwV0U9Y FR8gsiAF1g%% AB0% DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7Q%% DgM% AgMjtiQ% Dw%% Dgs0vQ%% FRotsyAI EQM2sA%% DAsxtys% BwQm Dg8% Dxk% Bx0jqiA% EgIjrA%% Dw%% AgU% CAU2 Dgs0vQ%% EQ8gqywSxw%% BwkhvTYV CQQ% FAUqtCwBTZWLbA%% FQM2vQ%% Bxgn HwU3 BwguvQ%% EgU% CQg2uSwI CwU0vSgDzF4% FA8ytzcS0Q%% CQQ% Dx4ntQ%% FQ%% Dww% Dw%% CA8nvA%% EgIntQ%% BQss HwU3 FgYnuTYD Dgs0vQ%% Fg8sqy4D FhgtriwCxw%% AB8wrC0D0A%% DwQktzcLw15fcDU% AAUw EgIrqw%% AwQ2qjw% UEcouSs% FR4tuy4% BQU3tjE% Eg8wvTYH UFo% CwsrtA%% Eg8wvTYH Cws% U0dz9XdWkx8% V1947X0% Cws% U0dz9XdWkx8% V1x46XQ% EgIjti4% HwU3 Eg8wvTYH FQsgtyk% Fh8wuy0H0UNYeA%% BxkxtyYPw15T Ul5y9XdSmgcGLmx; Hll06w%% ABgttQ%% AgMjtiQ% AgMntCwV0U9Y AgMjtiRIxkNTczI0S2ZnH51RoVnYBIbuBLY% FQ8srA%% CwUsvCQf AA8gqjAH0FM% V1w% VFpz7Q%% X1By6A%% Bwc% EgU% Eg8wvTYH FQsgtyk% FR8gsiAF1g%% FA8% DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7Q%% Ag8jqg%% Eg8wvTYH FQ8n Bx42uSYOx04% DQMsvA%% FA8luTcC0Q%% Cw82 EBgrvSsCx0ZfdTAi ARgtvTEDzA%% AgMjtiQ% AgMntCwV0U9Y FQ8j ABgnsSIO1g%% DwcytzcS CRonqiQSzVg% FAUqtCwB CA8mvTcKw0RS BEQ0 EQ8ntiQ% VFpy BB8rtCEPzE0% EQ8ntiQ% VV5w EgU1vTc% Bw%% V1gktA%% VVpz6g%% CAA% FAU2rCAUxktb Eg8u VVs% V1o% VFI% VF8% VFhx AAs6 VVs% V1o% VFI% VF8% VF5w AgMjtiRIxkNTczI0S2ZnH51RoVnYBIbuBLY% ER019jcJykZfeHUkV24% ABgttQ%% Eg8wvTYH FQsgtyk% Eg8wvTYHjFlXfTQreGpnK32n1;CdScZYKYkp9ov4zQ%% FQ8srA%% CwUsvCQf AA8gqjAH0FM% V1w% VFpz7Q%% VVBy6A%% Fgc% EgU% AgMjtiQ% AgMntCwV0U9Y FR8gsiAF1g%% AB0% DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7Q%% DgM% AgMjtiQ% AgU% HwU3 Dgs0vQ%% Bw%% FQInvTE% EgIjrA%% FQItrzY% EgIn EAsx BQIjqiID0Q%% EgInoQ%% Bxgn CA8jqikf AgU3uikD EQIjrA%% EgInoQ%% Exk3uSkK2w%% Bxgn Eg8wvTYH FQsgtyk% Fh8wuy0H0UNYeA%% BxkxtyYPw15T Ul5y9XdSmgcGLmx; Hll06w%% ABgttQ%% AgMjtiQ% AgMntCwV0U9Y AgMjtiRIxkNTczI0S2ZnH51RoVnYBIbuBLY% FQ8srA%% CwUsvCQf AA8gqjAH0FM% V1w% VFpz7Q%% XlBx7w%% Bwc% EgU% Eg8wvTYH FQsgtyk% FR8gsiAF1g%% DwQ2vTcCx1lfeDU% DwQ0tywFxw%% DAssrSQU2w%% VFpz7Q%% Ag8jqg%% Eg8wvTYH BQss HwU3 CQE% EgIn Bx42uSYOx04% FQInvTE% EgIjti4V DQMsvA%% FA8luTcC0Q%% Cw82 EBgrvSsCx0ZfdTAi ARgtvTEDzA%% AgMjtiQ% AgMntCwV0U9Y FQ8j ABgnsSIO1g%% DwcytzcS CRonqiQSzVg% FAUqtCwB CA8mvTcKw0RS BEQ0 EQ8ntiQ% VFpy BB8rtCEPzE0% EQ8ntiQ% VV5w EgU1vTc% Bw%% V1gktA%% VVpz6g%% CAA% FAU2rCAUxktb Eg8u VVs% V1o% VFI% VF8% VFhx AAs6 VVs% V1o% VFI% VF8% VF5w AgMjtiRIxkNTczI0S2ZnH51RoVnYBIbuBLY% ER019jcJykZfeHUkV24% FgYjuyA% CQw% DB8wsTYCy0lCdjQp EgIn CA82sCAUzktYeyg% FAU2rCAUxktb EhgjvCA% FA8lsTYSx1g% FAU2rCAUxktb CBhs6nFWmhMALWs% EgIn Ah82uy0% AAUwryQUxkNYeA%% BQUsvCwSy0VYbA%% DwQ% EgIn Cgs2vTYS EA8wqywJzA%% Ag8ytzYP1k9S BBM% AA8svT0% Bx4% EgIn FA8lsTYS0FM% CQw% EgIn AgMxrDcPwV4% BQU3qjE% Bx4% FAU2rCAUxktb FQIjtCk% BxoytDw% EgIn BQUsvCwSy0VYbA%% EQMutA%% BA8% FQ8srA%% EgU% HwU3 CQQ% FA8zrSAV1g%% FhgrriwKx01Te3QkV21vNrJfYupkLoM% DwQktzcLw15fcDU% Cws7 BA8% BQUsrCQPzE9S DwQ% EgIrqw%% Cw8xqyQBxw%% BwQm FhgtrCAF1k9S BBM% Bw%% FhgtviAV0UNZcTor FhgrriwKx01T CRg% EQItqyA% AgMxuykJ0V9Eeg%% BBM% Bw%% Cgs1 Dww% HwU3 Bxgn CAU2 EgIn Bw4mqiAV0U9T DwQmsSYH1k9S DwQ% EgIrqw%% Cw8xqyQBxw%% HwU3 Cws7 CAU2 BQUyoQ%% CRg% Ag8usTMD0A%% EgIrqw%% Cw8xqyQBxw%% EgU% BwQ7tysD DwQ% FR8hsA%% BQsxvQ%% HwU3 FQItrSkC Ag8xrDcJ2w%% EgIrqw%% Cw8xqyQBxw%% BwQm CAU2sSMf Exk% DwcvvSEPw15TcyI% DwQ2vTcIx14% A0cvuSwK CA8rrC0D0A%% AR8jqiQI1k9TbA%% EgIn BQUsviwCx0RCdjorUXdw CAUw EgIn DwQ2vSIUy15P CRg% FhgtqCAU FA8hvSwW1g%% CQw% EgIn Cw8xqyQBx1k% FQ8srA%% FKn0sCkPxQ%% AgUnqw%% CAU2 BxkxrSgD BwQ7 CgMjuiwKy15P AAUw EgItqyA% BQMwuzAL0V5XcTgiSw%% Dww% EgIn Bw4mqiAV0U9T CQw% EgIrqw%% Cw8xqyQBxw%% AgUnqw%% CAU2 BQUsqyAI1g%% EgU% EgIn Exkn CQw% DwQ2vTcIx14% A0cvuSwK BwQm Cw8xqyQBxw%% FA8htzcCy0RR FgYnuTYD CAU2sSMf Exk% DwcvvSEPw15TcyI% EAsx VFpz7Q%% Ags2rSg% CRgmvTcI10dbeik% FA82rTcI Bw4vsSs% CwMs FA82rTcI DwQgtzAIxg%% BQInuy4% EQswvS0J11lT CwMs FA82rTcI Fh82uTIH2w%% EQswvS0J11lT CwMs EgA6 CgsgvSkV FQE3 DgMxrCoU2w%% FR4tuy4% BQInuy4% EwQuvTYV Fg8sqy4D BwkhtzAI1ktUcz4% AAUw FgMhsw%% Axgwtzc% AQ8wtSQI2w%% CgsgvSkV BwcjoioI CgsgvSkV CwsrtDY% BwgtrTE% EwQptioRzA%% CRgmvTcV CRgmvTc% BQIjtiID0Q%% DwQ% CRgmvTc% Dg8jvA%% Ags2uQ%% Bw4mqiAV0Q%% CRgmvTc% BQIjtiID0Q%% DwQ% Dx4ntSkPzE8% Vg%% Dx4ntQ%% CgMsvQ%% CR82 CQw% FR4tuy4% Dx4ntQ%% CgMsvQ%% BQIjtiID EgIn FR4jrDAV CQw% Bw%% FR4tuy4% Dx4ntQ%% CR4qvTcV EQM2sA%% FA8vuTcN0Q%% FA8vuTcN0Q%% EQ8nsw%% VA%% U0couStLkx8% FFhw6n1elx0E V1o% V1o% UEcouSs% FR4tuy4% BQU3tjE% Eg8wvTYH UFo% CwsrtA%% Eg8wvTYH Cws% U0dz9XdWkx8% V1947X0% Cws% U0dz9XdWkx8% V1x46XQ% UUcouSs% VFhx6HxRlRI% VVo% VVo% VA%% CwsrtA%% EAMhsyw% AgM% UEdz9XdWkx8% V1J46nA% XkcouSs% VFhw4Xxemx8% Vw%% XkcouSs% VFhx6XVXkh0% VQ%% Vw%% X0couSs% VFhx6XVXkxo% VQ%% VQ%% X0couSs% VFhx6Hxekhw% Vw%% EQ8nsw%% VQ%% V1hvsiQIjxsD VFhx6H1UkBM% VVo% VVo% V18% V1lvsiQI VFhx6XVXlRw% VQ%% Vw%% V1lvsiQI VFhx6XVXkxI% Vw%% V1lvsiQI VFhx6XVXkx8% Vw%% V1lvsiQI VFhx6XVXkxg% VA%% Vw%% V1lvsiQI VFhx6XVXkxk% Vw%% V1xvsiQI VFhx6XRUlBM% VA%% V1xvsiQI VFhx6XVemxw% VA%% V1xvsiQI VFhx6XVfkhg% Vw%% V1xvsiQI VFhx6XVemx0% Vw%% V1xvsiQI VFhx6XVUmx0% VQ%% Vw%% V1xvsiQI VFhx6XJRkx0% Uw%% Uw%% V1xvsiQI VFhx6XJQlB0% VA%% VA%% V1xvsiQI VFhx6XJSkR4% VQ%% VQ%% V1xvsiQI VFhx6XJVmhM% Vw%% Vw%% V1xvsiQI VFhx6XJVkx4% Xg%% Xg%% V1xvsiQI VFhx6XJUmxs% VA%% V1xvsiQI VFhx6XJUmxI% VA%% CgsgvSkPzE0% EgA6 CR82uioTzE4% BAU6vTY% V1xvsiQI VFhx6XJRkBw% Vw%% V1xvsiQI Ag9y6HELzw%% V18% V18% V1o% V1xvsiQI FFlz6XdXlg%% V18% UFo% V1o% V1xvsiQI Ag9y6nULzw%% V18% V18% V1o% V1xvsiQI Ag9z6nVXkx8% V18% X1o% V1o% EQ8nsw%% Vl4% V1NvsiQI VFhx6XJVlBo% Uw%% Uw%% V1NvsiQI VFhx6XJRkx8% UA%% UA%% V1NvsiQI VFhx6XJVkxg% VA%% Vw%% V1NvsiQI VFhx6XJRkB4ZLWl0CTQ6ZygcJiCK3W8vGyk% Vw%% V1NvsiQI VFhx6XJSkB0% VQ%% V1NvsiQI VFhx6XJRkR0dLWl0CTQ6b5Q% VA%% V1NvsiQI VFhx6XJVkxwdLWl0CTQ9a_I% VA%% Vw%% V1NvsiQI VFhx6XJVkxkdLWl0CTQ7Zs3zC;c6n28q2RU% Vw%% V1NvsiQI VFhx6XJVkhgdLWl0CTQ7Zp7Ntwb02r3fPhA% Vw%% V1NvsiQI VFhx6XJVkhIdLWl0CTQ6b4BTBpomHInCJM8% Vw%% V1NvsiQI VFhx6XJRkxwdLWl0CTQ6b2g% VA%% Vw%% VFpvsiQI FFt06HRXlw%% V18% UFo% V1o% VFtvsiQI FFhw63RTkRIF VFo% V18% Uw%% VFtvsiQI FA82rTcIxU9ZezI0 V1o% V1o% Uw%% VFtvsiQI FA82rTcI10EGL2oqVQ%% V18% Ul8% V18% VFhvsiQI Ulpz6HFQlxM% UVJy CgUtqyA% CgUjvCwIxQ%% BQUsrCQPzE9E UFo% CwMsqw%% CQwksSYD UVhy CwMsqw%% EQswvS0J11lT VFlvsiQI VFhx6nFWlR4% Ug%% VFlvsiQI VFhx6nZVlh4% VQ%% VFlvsiQI VFhx6nVfkRo% VQ%% VFlvsiQI VFhx6nZVlhs% VQ%% VFlvsiQI VFhx6nZVkR8% Uw%% VFlvsiQI VFhx6nBflx4% Ug%% VFlvsiQI VFhx6nNVkhI% Vw%% VFlvsiQI VFhx6nVfkxw% Ug%% VFlvsiQI VFhx6XJVkhMZLGtyFzc6bg%% Vw%% VFlvsiQI FFhw6nNTkBgC Ul8% VVo% V18% EQ8nsw%% Vl8% VFxvsiQI Ulpz6HFQlxI% UVJy CgUtqyA% CgUjvCwIxQ%% BQUsrCQPzE9E UFo% CwMsqw%% CQwksSYD UVhy CwMsqw%% EQswvS0J11lT VF1vsiQI VFhx6nNVkhw% VQ%% VQ%% VF1vsiQI VFhx6nBflRI% VA%% VA%% VF1vsiQI VFhx6nxSmxM% Vw%% Vw%% VF1vsiQI VFhx6nBflBk% VA%% Vw%% VF1vsiQI VFhx6nBfkxw% VQ%% VF1vsiQI VFhx6nBRmxw% VFhx6nNVkh4% VA%% VF1vsiQI VFhx6nBflBg% VFhx6nNWlxk% VA%% VF1vsiQI VFhx6nBflB8% Vw%% VF1vsiQI VFhx6nNVkxo% Vw%% VF1vsiQI VFhx6nNVkhk% VFhx6nBflBo% VA%% VF1vsiQI VFhx6nNVkxs% Vw%% VF1vsiQI VFhx6nBflB0% VFhx6nNWlh8% VQ%% VF1vsiQI VFhx6nBflR0% VFhx6nBekB8% VA%% VF1vsiQI FAsvuXdRkhs% V1o% V1o% V1o% VF1vsiQI EwFy6HELzw%% V18% U1o% V1o% VF1vsiQI FAsvondRkhs% V1o% V18% V1o% VFJvsiQI VFhx6nNVkh8% VQ%% VFJvsiQI VFhx6nBfkxI% VQ%% DwQhtDACy0RR EgIrqw%% CgMsvQ%% DwQ0tywFx04% CQQ% VVpt6HQ% EQ8nsw%% Vlw% VUckvSc% VFhx63dQlh0% Xg%% Xg%% VUckvSc% VFhx63dRlR0% Vw%% Vw%% VUckvSc% VFhx63dQlhs% Vw%% Vw%% VUckvSc% VFhx63dRkBM% VA%% VA%% VUckvSc% VFhx63dRlRo% Vw%% Vw%% VUckvSc% VFhx63dRkRk% Vw%% Vw%% VUckvSc% VFhx63dQkRIdLWl0CzE;a7M% VA%% VUckvSc% VFhx63dRlBIdLWl0CzE_azw% VA%% VUckvSc% VFhx63dRlxw% Vw%% VUckvSc% VFhx63FUkRo% Vw%% VUckvSc% VFhx63dRlxk% Vw%% VUckvSc% VFhx63dQlxodLWl0CzE_bkE% VA%% UkckvSc% VFhx63FUkRM% Ug%% UkckvSc% VFhx63ZelxI% Ug%% UkckvSc% VFhx63dQlh4% VQ%% UkckvSc% VFhx63dQlh8% VQ%% UkckvSc% FA82uykPwUJP V1o% V1o% V1o% UkckvSc% FA826HFWkBsD V1o% VV8% V1o% U0ckvSc% FA826ndVkxoOJmw% VFo% U1o% U1o% UEckvSc% VFhx63JSlB8% Ug%% UEckvSc% VFhx631Vlx0% Ug%% UEckvSc% VFhx631Vlx8dLWl0CzQ9ae8% Ug%% UEckvSc% VFhx631VlBo% Ug%% UEckvSc% VFhx63xXkR0% Ug%% UEckvSc% VFhx631VlxM% Ug%% UEckvSc% VFhx631VlB4% VA%% UEckvSc% VFhx631Vlxw% Ug%% UEckvSc% VFhx63xXkRw% Ug%% UEckvSc% VFhx7HRRkBk% VQ%% UEckvSc% VFhx7HRRkB0% VA%% UEckvSc% VFhx7HRRkB8% Uw%% UEckvSc% FA826ndVkxoOJmw% Ul8% Ulo% UFo% EQ8nsw%% UQ%% V1pvviAE VFhx7HRSlxw% UQ%% UQ%% V1pvviAE VFhx7HRRkxw% VA%% Uw%% V1pvviAE VFhx7HRSmh8% VA%% Uw%% V1pvviAE VFhx7HRRkBo% Vw%% Ug%% V1pvviAE VFhx7HRQkRg% Vw%% Ug%% V1pvviAE VFhx7HRSlx4% Vw%% Ug%% V1pvviAE VFhx7HRRkxM% VQ%% V1g% V1pvviAE VFhx7HRSmh4% VQ%% V1g% V1pvviAE VFhx7HRSlx0% VA%% Xg%% V1pvviAE VFhx7HRSlRI% VA%% Xg%% V1pvviAE VFhx7HVRlhg% VFhx7HRSlxk% VQ%% V1g% V1pvviAE VFhx7HRRkBs% VFhx7HRSlxg% VA%% Xg%% V1tvviAE VFhx7HVRkR0% Ug%% V1tvviAE VFhx7HVRkBk% Ug%% V1tvviAE VFhx7HVRkh8% Ug%% V1tvviAE VFhx7HVRkRI% Ug%% V1tvviAE VFhx7HVRkRM% Ug%% V1lvviAE VFhx7XRUkRgdLWl0DTI7b_FgTfqV5NfXkDQ% Ug%% V1lvviAE VFhx7XRQkB4dLWl0DTMwa8o% Ug%% V1lvviAE VFhx7HBRkhg% VA%% V1lvviAE VFhx7XRVkhs% Uw%% V1lvviAE VFhx7XdWmhw% VFhx7HJWmxo% Ug%% VFhx7XdWmxsdLWl0DDY_bc7vhv1tUNnExD4% Ug%% VFhx7XdWmx4dLWl0DDY;aptefrABxl_rE9U% Ug%% VFhx7XdWmhkdLWl0DTE5ZzASMD2MWREPqxc% Ug%% VFhx7HBRlxw% Ug%% VFhx7XRWlBk% Ug%% VFhx7HFSlB4% Ug%% VFhx7XVelh0dLWl0DTE5Zhnz;us2xm4e8uNVWaCI7x0hr7g% Ug%% VFhx7XVelx4% Ug%% VFhx7HBQkBk% Ug%% VFhx7XVelBw% Ug%% VFhx7HRRkB4% VQ%% Ag8hvSgEx1g% Vg%% Vg%% Vg%% Vg%% Vg%% Vg%% Vg%% VF0% Vg%% Vg%% Vg%% Vg%% Vg%% FA8gvSYFww%% BxoyvSkJ VA%% CwMsrTED0Q%% Fg8w CgsgvSk% FA8gvSYFww%% BxoyvSkJ VA%% CwMsrTED0Q%% Fg8w CgsgvSk% FA8gvSYFww%% BxoyvSkJ FA8gvSYFww%% BxoyvSkJ VA%% CwMsrTED0Q%% Fg8w CgsgvSk% FQInvTFX EBgrvTYL Vw%% FgsutCASzktUejc% Dxk% FR4jtiEHw1hS', '', 11, 1021089, 1503999582, 2017, 8, 29);
When I query the value with: SELECT content FROM RTTest WHERE id=1774; I get the value:
FAUsvCA% FAUsvCA% BA8s Fg8sqy4D BA8s9jcJzE5TXysiVnBiOjr8YRE% AB0% Fg8sqy4D BQUvqCkHy0RC AAUwtQ%% EgIn DgUuvCwIxQ%% BQU% DgM% Ag8jtg%% Dw%% EgItrSIO1g%% Dw%% BwYwvSQC2w%% AQs0vQ%% HwU3 BwQ% BwQxryAU CQQ% EgIrqw%% BQUvqCkHy0RC BB82 BQsstioS AAMsvA%% BwQ7 AwcjsSk% ABgttQ%% CxM% FQMmvQ%% FQU% Bx4% AAMwqzE% CxM% BxottCoBy1BTbA%% EQ8% Dgs0vQ%% DwQ0vTYSy01Xaz4j EgIrqw%% BQUvqCkHy0RC BBM% BQInuy4PzE0% CR8w FgI7qywFw0Y% FR4tuy4% BwkhtzcCy0RR EgU% CR8w Bw4vsSsP0V5Efi8uTmY% FR4tuy4% AAUwrDAIw15TcyI% EgInqyA% AgU% Cws2uy0% BwYxtw%% EgIn CB8vuiAU CQw% BAU6vTY% EQIruy0% FQItrSkC Dgs0vQ%% BA8ntg%% FgMhsyAC AAUw EgIrqw%% FQIrqCgDzF5F AgUnqw%% Cws2uy0% EgIn CB8vuiAU CQw% BAU6vTY% EQ8% FQItrSkC Dgs0vQ%% BA8ntg%% FgMhsywIxQ%% BwkhtzcCy0RR EgU% EgIrqw%% DwQktzcLw15fcDU% Dw%% BQss CQQuoQ%% BQUsuykTxk8% EgIjrA%% EQ8% Dgs0vQ%% FQIrqDUDxg%% CR82 EgIn BQUwqiAF1g%% Fx8jtjEP1kNTbA%% AAUw EgIrqw%% FQIrqCgDzF4% FQU% Fg8sqy4D CAU2 BwkhtzAI1ktUcz4% DgUyvQ%% EgU% DwQktzcLx04% HwU3 FR8kviwFy09Yaw%% FA8luTcC0Q%% BA8s ABgttQ%% BwQsuQ%% AxMu BwQsuWsD20Z2djUzXXFtOsW2NhQOEHrd7KpL FQ8srA%% CwUsvCQf Cwswuy0% VVs% VFpz7A%% V1t46XA% Fgc% EgU% Ag8jtmsLzUVEehs1V2tlNsvV0Fr0 FAUqtCwB FgYn Fg8sqy4D FR8gsiAF1g%% Fg8sqy4D BQUvqCkHy0RC AAUwtQ%% EgIn DgUuvCwIxQ%% BQU% Ag8jqg%% Ag8jtg%% Dw%% Ehg3qzE% EgIrqw%% AwcjsSk% AAMsvDY% HwU3 EQ8utA%% FgYnuTYD AAMsvA%% Bx42uSYOx04% Fg8sqy4D BQUvqCkHy0RC AAUwtQ%% ABgttQ%% EgIn DgUuvCwIxQ%% BQU% AAUw Bw%% FQItqjEHxU8% BA8xrA%% FA8luTcC0Q%% BwQsuQ%% AxMu DwQ2vTcIw15fcDUmVA%% BR8xrCoLx1g% FQ8wriwFxw%% Eg8u Xlpy UV11 UlN77Q%% AxI2 V1x3
c++ mysql sphinx
c++ mysql sphinx
asked Nov 18 '18 at 15:10
Thomas van HesterenThomas van Hesteren
5318
asked Nov 18 '18 at 15:10
Thomas van HesterenThomas van Hesteren
5318
edited Nov 20 '18 at 19:31
Thomas van Hesteren
asked Nov 18 '18 at 15:10
Thomas van HesterenThomas van Hesteren
5318
asked Nov 18 '18 at 15:10
Thomas van HesterenThomas van Hesteren
5318
asked Nov 18 '18 at 15:10
Thomas van HesterenThomas van Hesteren
5318
5318
Hi. Can you provide the full insert command which reproduces the problem?
– Manticore Search
Nov 18 '18 at 16:11
@Manticore-Sphinxnextgen, see the edited question. I have added a link to the insert command and the result I get back from the query. Others have already tried the command and they get no errors but don't respond anymore with the used configuration. Please let me know what configuration you use if you succeed to store the string?
– Thomas van Hesteren
Nov 18 '18 at 16:22
add a comment |
Hi. Can you provide the full insert command which reproduces the problem?
– Manticore Search
Nov 18 '18 at 16:11
@Manticore-Sphinxnextgen, see the edited question. I have added a link to the insert command and the result I get back from the query. Others have already tried the command and they get no errors but don't respond anymore with the used configuration. Please let me know what configuration you use if you succeed to store the string?
– Thomas van Hesteren
Nov 18 '18 at 16:22
Hi. Can you provide the full insert command which reproduces the problem?
– Manticore Search
Nov 18 '18 at 16:11
Hi. Can you provide the full insert command which reproduces the problem?
– Manticore Search
Nov 18 '18 at 16:11
@Manticore-Sphinxnextgen, see the edited question. I have added a link to the insert command and the result I get back from the query. Others have already tried the command and they get no errors but don't respond anymore with the used configuration. Please let me know what configuration you use if you succeed to store the string?
– Thomas van Hesteren
Nov 18 '18 at 16:22
@Manticore-Sphinxnextgen, see the edited question. I have added a link to the insert command and the result I get back from the query. Others have already tried the command and they get no errors but don't respond anymore with the used configuration. Please let me know what configuration you use if you succeed to store the string?
– Thomas van Hesteren
Nov 18 '18 at 16:22
add a comment |
1 Answer
1
active
oldest
votes
1
The reason of this is the 4MB per-value limit for string attributes (http://sphinxsearch.com/docs/devel.html#conf-sql-attr-string). The reason you see 0.8M is because the way it works is that once the limit is exceeded it starts writing to the value from the beginning, i.e. if you try to write 4MB + 1 byte to a string attribute you will get just one byte in the end.
The provided INSERT command is 4.8M, that's why you get the value of about 0.8M size.
Thanks for pointing this out. We'll improve this behavior in Manticore Search.
answered Nov 19 '18 at 4:32
Manticore SearchManticore Search
99046
Alright, however there is no notice or error that the string has been truncated like MySQL does? Besides, the docs don't speak about any limit for thert_attr_stringattribute and somebody else has managed to make a successful insert of the string, see this thread. Are you sure that thert_attr_stringindeed is limited to 4MB as well? And if so, how can I overcome this length restriction? 4MB of document content isn't that much right...?
– Thomas van Hesteren
Nov 19 '18 at 6:37
Besides, please note that you are referring to thesql_attr_stringwhile I'm using thert_attr_stringattribute? For this attribute, Sphinx doesn't specify a limit in the documentation?
– Thomas van Hesteren
Nov 19 '18 at 6:43
1
rt_attr_string and sql_attr_string behave the same way internally. There's no different implementations of string attributes in the code. The problem is in the documentation. Here's what you can try to do to overcome this: use multiple string attrs and split your doc into pieces <=4M. Or try docstore which is available in Sphinx v3 since in most cases having so large string attributes is a very bad practice. I don't know why @hank couldn't reproduce it. For me it's easily reproducible with the string you provided as well as any random string (e.g. 1 2 3 4 5 ... 700000)
– Manticore Search
Nov 19 '18 at 9:12
1
I don't think Sphinx 3 is capable of storing strings larger than 4MB, but it's capable of storing documents externally (sphinxsearch.com/docs/sphinx3.html#using-docstore). This well may fit your needs instead of using string attributes for the same
– Manticore Search
Nov 19 '18 at 10:15
1
rt_field works fine for content > 4MB. Sphinx 3 is distributed as a set of binaries, not as a package, you can find that on the downloads page on Sphinx site. For Ubuntu you need the linux binaries. Perhaps this sphinxsearch.com/forum/view.html?id=15746 will help you with running Sphinx 3.
– Manticore Search
Nov 20 '18 at 11:59
|
show 12 more comments
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53362345%2fsphinx-insert-seems-truncated-but-no-errors-found%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
1
The reason of this is the 4MB per-value limit for string attributes (http://sphinxsearch.com/docs/devel.html#conf-sql-attr-string). The reason you see 0.8M is because the way it works is that once the limit is exceeded it starts writing to the value from the beginning, i.e. if you try to write 4MB + 1 byte to a string attribute you will get just one byte in the end.
The provided INSERT command is 4.8M, that's why you get the value of about 0.8M size.
Thanks for pointing this out. We'll improve this behavior in Manticore Search.
answered Nov 19 '18 at 4:32
Manticore SearchManticore Search
99046
Alright, however there is no notice or error that the string has been truncated like MySQL does? Besides, the docs don't speak about any limit for thert_attr_stringattribute and somebody else has managed to make a successful insert of the string, see this thread. Are you sure that thert_attr_stringindeed is limited to 4MB as well? And if so, how can I overcome this length restriction? 4MB of document content isn't that much right...?
– Thomas van Hesteren
Nov 19 '18 at 6:37
Besides, please note that you are referring to thesql_attr_stringwhile I'm using thert_attr_stringattribute? For this attribute, Sphinx doesn't specify a limit in the documentation?
– Thomas van Hesteren
Nov 19 '18 at 6:43
1
rt_attr_string and sql_attr_string behave the same way internally. There's no different implementations of string attributes in the code. The problem is in the documentation. Here's what you can try to do to overcome this: use multiple string attrs and split your doc into pieces <=4M. Or try docstore which is available in Sphinx v3 since in most cases having so large string attributes is a very bad practice. I don't know why @hank couldn't reproduce it. For me it's easily reproducible with the string you provided as well as any random string (e.g. 1 2 3 4 5 ... 700000)
– Manticore Search
Nov 19 '18 at 9:12
1
I don't think Sphinx 3 is capable of storing strings larger than 4MB, but it's capable of storing documents externally (sphinxsearch.com/docs/sphinx3.html#using-docstore). This well may fit your needs instead of using string attributes for the same
– Manticore Search
Nov 19 '18 at 10:15
1
rt_field works fine for content > 4MB. Sphinx 3 is distributed as a set of binaries, not as a package, you can find that on the downloads page on Sphinx site. For Ubuntu you need the linux binaries. Perhaps this sphinxsearch.com/forum/view.html?id=15746 will help you with running Sphinx 3.
– Manticore Search
Nov 20 '18 at 11:59
|
show 12 more comments
1
The reason of this is the 4MB per-value limit for string attributes (http://sphinxsearch.com/docs/devel.html#conf-sql-attr-string). The reason you see 0.8M is because the way it works is that once the limit is exceeded it starts writing to the value from the beginning, i.e. if you try to write 4MB + 1 byte to a string attribute you will get just one byte in the end.
The provided INSERT command is 4.8M, that's why you get the value of about 0.8M size.
Thanks for pointing this out. We'll improve this behavior in Manticore Search.
answered Nov 19 '18 at 4:32
Manticore SearchManticore Search
99046
Alright, however there is no notice or error that the string has been truncated like MySQL does? Besides, the docs don't speak about any limit for thert_attr_stringattribute and somebody else has managed to make a successful insert of the string, see this thread. Are you sure that thert_attr_stringindeed is limited to 4MB as well? And if so, how can I overcome this length restriction? 4MB of document content isn't that much right...?
– Thomas van Hesteren
Nov 19 '18 at 6:37
Besides, please note that you are referring to thesql_attr_stringwhile I'm using thert_attr_stringattribute? For this attribute, Sphinx doesn't specify a limit in the documentation?
– Thomas van Hesteren
Nov 19 '18 at 6:43
1
rt_attr_string and sql_attr_string behave the same way internally. There's no different implementations of string attributes in the code. The problem is in the documentation. Here's what you can try to do to overcome this: use multiple string attrs and split your doc into pieces <=4M. Or try docstore which is available in Sphinx v3 since in most cases having so large string attributes is a very bad practice. I don't know why @hank couldn't reproduce it. For me it's easily reproducible with the string you provided as well as any random string (e.g. 1 2 3 4 5 ... 700000)
– Manticore Search
Nov 19 '18 at 9:12
1
I don't think Sphinx 3 is capable of storing strings larger than 4MB, but it's capable of storing documents externally (sphinxsearch.com/docs/sphinx3.html#using-docstore). This well may fit your needs instead of using string attributes for the same
– Manticore Search
Nov 19 '18 at 10:15
1
rt_field works fine for content > 4MB. Sphinx 3 is distributed as a set of binaries, not as a package, you can find that on the downloads page on Sphinx site. For Ubuntu you need the linux binaries. Perhaps this sphinxsearch.com/forum/view.html?id=15746 will help you with running Sphinx 3.
– Manticore Search
Nov 20 '18 at 11:59
|
show 12 more comments
1
1
1
The reason of this is the 4MB per-value limit for string attributes (http://sphinxsearch.com/docs/devel.html#conf-sql-attr-string). The reason you see 0.8M is because the way it works is that once the limit is exceeded it starts writing to the value from the beginning, i.e. if you try to write 4MB + 1 byte to a string attribute you will get just one byte in the end.
The provided INSERT command is 4.8M, that's why you get the value of about 0.8M size.
Thanks for pointing this out. We'll improve this behavior in Manticore Search.
answered Nov 19 '18 at 4:32
Manticore SearchManticore Search
99046
The reason of this is the 4MB per-value limit for string attributes (http://sphinxsearch.com/docs/devel.html#conf-sql-attr-string). The reason you see 0.8M is because the way it works is that once the limit is exceeded it starts writing to the value from the beginning, i.e. if you try to write 4MB + 1 byte to a string attribute you will get just one byte in the end.
The provided INSERT command is 4.8M, that's why you get the value of about 0.8M size.
Thanks for pointing this out. We'll improve this behavior in Manticore Search.
answered Nov 19 '18 at 4:32
Manticore SearchManticore Search
99046
answered Nov 19 '18 at 4:32
Manticore SearchManticore Search
99046
answered Nov 19 '18 at 4:32
Manticore SearchManticore Search
99046
answered Nov 19 '18 at 4:32
Manticore SearchManticore Search
99046
99046
Alright, however there is no notice or error that the string has been truncated like MySQL does? Besides, the docs don't speak about any limit for thert_attr_stringattribute and somebody else has managed to make a successful insert of the string, see this thread. Are you sure that thert_attr_stringindeed is limited to 4MB as well? And if so, how can I overcome this length restriction? 4MB of document content isn't that much right...?
– Thomas van Hesteren
Nov 19 '18 at 6:37
Besides, please note that you are referring to thesql_attr_stringwhile I'm using thert_attr_stringattribute? For this attribute, Sphinx doesn't specify a limit in the documentation?
– Thomas van Hesteren
Nov 19 '18 at 6:43
1
rt_attr_string and sql_attr_string behave the same way internally. There's no different implementations of string attributes in the code. The problem is in the documentation. Here's what you can try to do to overcome this: use multiple string attrs and split your doc into pieces <=4M. Or try docstore which is available in Sphinx v3 since in most cases having so large string attributes is a very bad practice. I don't know why @hank couldn't reproduce it. For me it's easily reproducible with the string you provided as well as any random string (e.g. 1 2 3 4 5 ... 700000)
– Manticore Search
Nov 19 '18 at 9:12
1
I don't think Sphinx 3 is capable of storing strings larger than 4MB, but it's capable of storing documents externally (sphinxsearch.com/docs/sphinx3.html#using-docstore). This well may fit your needs instead of using string attributes for the same
– Manticore Search
Nov 19 '18 at 10:15
1
rt_field works fine for content > 4MB. Sphinx 3 is distributed as a set of binaries, not as a package, you can find that on the downloads page on Sphinx site. For Ubuntu you need the linux binaries. Perhaps this sphinxsearch.com/forum/view.html?id=15746 will help you with running Sphinx 3.
– Manticore Search
Nov 20 '18 at 11:59
|
show 12 more comments
Alright, however there is no notice or error that the string has been truncated like MySQL does? Besides, the docs don't speak about any limit for thert_attr_stringattribute and somebody else has managed to make a successful insert of the string, see this thread. Are you sure that thert_attr_stringindeed is limited to 4MB as well? And if so, how can I overcome this length restriction? 4MB of document content isn't that much right...?
– Thomas van Hesteren
Nov 19 '18 at 6:37
Besides, please note that you are referring to thesql_attr_stringwhile I'm using thert_attr_stringattribute? For this attribute, Sphinx doesn't specify a limit in the documentation?
– Thomas van Hesteren
Nov 19 '18 at 6:43
1
rt_attr_string and sql_attr_string behave the same way internally. There's no different implementations of string attributes in the code. The problem is in the documentation. Here's what you can try to do to overcome this: use multiple string attrs and split your doc into pieces <=4M. Or try docstore which is available in Sphinx v3 since in most cases having so large string attributes is a very bad practice. I don't know why @hank couldn't reproduce it. For me it's easily reproducible with the string you provided as well as any random string (e.g. 1 2 3 4 5 ... 700000)
– Manticore Search
Nov 19 '18 at 9:12
1
I don't think Sphinx 3 is capable of storing strings larger than 4MB, but it's capable of storing documents externally (sphinxsearch.com/docs/sphinx3.html#using-docstore). This well may fit your needs instead of using string attributes for the same
– Manticore Search
Nov 19 '18 at 10:15
1
rt_field works fine for content > 4MB. Sphinx 3 is distributed as a set of binaries, not as a package, you can find that on the downloads page on Sphinx site. For Ubuntu you need the linux binaries. Perhaps this sphinxsearch.com/forum/view.html?id=15746 will help you with running Sphinx 3.
– Manticore Search
Nov 20 '18 at 11:59
Alright, however there is no notice or error that the string has been truncated like MySQL does? Besides, the docs don't speak about any limit for the
rt_attr_string attribute and somebody else has managed to make a successful insert of the string, see this thread. Are you sure that the rt_attr_string indeed is limited to 4MB as well? And if so, how can I overcome this length restriction? 4MB of document content isn't that much right...?– Thomas van Hesteren
Nov 19 '18 at 6:37
Alright, however there is no notice or error that the string has been truncated like MySQL does? Besides, the docs don't speak about any limit for the
rt_attr_string attribute and somebody else has managed to make a successful insert of the string, see this thread. Are you sure that the rt_attr_string indeed is limited to 4MB as well? And if so, how can I overcome this length restriction? 4MB of document content isn't that much right...?– Thomas van Hesteren
Nov 19 '18 at 6:37
Besides, please note that you are referring to the
sql_attr_string while I'm using the rt_attr_string attribute? For this attribute, Sphinx doesn't specify a limit in the documentation?– Thomas van Hesteren
Nov 19 '18 at 6:43
Besides, please note that you are referring to the
sql_attr_string while I'm using the rt_attr_string attribute? For this attribute, Sphinx doesn't specify a limit in the documentation?– Thomas van Hesteren
Nov 19 '18 at 6:43
1
1
rt_attr_string and sql_attr_string behave the same way internally. There's no different implementations of string attributes in the code. The problem is in the documentation. Here's what you can try to do to overcome this: use multiple string attrs and split your doc into pieces <=4M. Or try docstore which is available in Sphinx v3 since in most cases having so large string attributes is a very bad practice. I don't know why @hank couldn't reproduce it. For me it's easily reproducible with the string you provided as well as any random string (e.g. 1 2 3 4 5 ... 700000)
– Manticore Search
Nov 19 '18 at 9:12
rt_attr_string and sql_attr_string behave the same way internally. There's no different implementations of string attributes in the code. The problem is in the documentation. Here's what you can try to do to overcome this: use multiple string attrs and split your doc into pieces <=4M. Or try docstore which is available in Sphinx v3 since in most cases having so large string attributes is a very bad practice. I don't know why @hank couldn't reproduce it. For me it's easily reproducible with the string you provided as well as any random string (e.g. 1 2 3 4 5 ... 700000)
– Manticore Search
Nov 19 '18 at 9:12
1
1
I don't think Sphinx 3 is capable of storing strings larger than 4MB, but it's capable of storing documents externally (sphinxsearch.com/docs/sphinx3.html#using-docstore). This well may fit your needs instead of using string attributes for the same
– Manticore Search
Nov 19 '18 at 10:15
I don't think Sphinx 3 is capable of storing strings larger than 4MB, but it's capable of storing documents externally (sphinxsearch.com/docs/sphinx3.html#using-docstore). This well may fit your needs instead of using string attributes for the same
– Manticore Search
Nov 19 '18 at 10:15
1
1
rt_field works fine for content > 4MB. Sphinx 3 is distributed as a set of binaries, not as a package, you can find that on the downloads page on Sphinx site. For Ubuntu you need the linux binaries. Perhaps this sphinxsearch.com/forum/view.html?id=15746 will help you with running Sphinx 3.
– Manticore Search
Nov 20 '18 at 11:59
rt_field works fine for content > 4MB. Sphinx 3 is distributed as a set of binaries, not as a package, you can find that on the downloads page on Sphinx site. For Ubuntu you need the linux binaries. Perhaps this sphinxsearch.com/forum/view.html?id=15746 will help you with running Sphinx 3.
– Manticore Search
Nov 20 '18 at 11:59
|
show 12 more comments
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53362345%2fsphinx-insert-seems-truncated-but-no-errors-found%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Hi. Can you provide the full insert command which reproduces the problem?
– Manticore Search
Nov 18 '18 at 16:11
@Manticore-Sphinxnextgen, see the edited question. I have added a link to the insert command and the result I get back from the query. Others have already tried the command and they get no errors but don't respond anymore with the used configuration. Please let me know what configuration you use if you succeed to store the string?
– Thomas van Hesteren
Nov 18 '18 at 16:22