Mixing
Large, sparse transition matrix - strategies for computing given memory limitations
$begingroup$
Beyond some size of the transition matrix, my computer cannot cope with the Markov Chain problem I am working on (in MATLAB). However, I am sure I am not aware of all the useful tricks that can extend the size of what is possible.
What are some must-know algorithms for working with a very large transition matrix?
A little more specifically, I am working on a problem about the transitions of a group from state to state, where the current state is defined by the configuration of elements in the group (think 1's and 0's for each element, although the state of each element may be more refined) and the transition depends on how the elements in the group change (between 1 and 0 in the simplest case)
Since the combinatorics very quickly blows-up when the group contain many elements and when we move beyond the simple 1 and 0 possible element states, I am having a difficult time examining the behaviour of the group for anything but small groups and simple possible states of elements.
I would like to examine large system behaviour, and therefore I am asking for strategies to push the limits.
It may also be important to note that some states of the group are absorbing states (in which elements in the group stop changing).
markov-chains markov-process
asked Jan 21 at 21:56
user120911user120911
231110
$endgroup$
add a comment |
$begingroup$
Beyond some size of the transition matrix, my computer cannot cope with the Markov Chain problem I am working on (in MATLAB). However, I am sure I am not aware of all the useful tricks that can extend the size of what is possible.
What are some must-know algorithms for working with a very large transition matrix?
A little more specifically, I am working on a problem about the transitions of a group from state to state, where the current state is defined by the configuration of elements in the group (think 1's and 0's for each element, although the state of each element may be more refined) and the transition depends on how the elements in the group change (between 1 and 0 in the simplest case)
Since the combinatorics very quickly blows-up when the group contain many elements and when we move beyond the simple 1 and 0 possible element states, I am having a difficult time examining the behaviour of the group for anything but small groups and simple possible states of elements.
I would like to examine large system behaviour, and therefore I am asking for strategies to push the limits.
It may also be important to note that some states of the group are absorbing states (in which elements in the group stop changing).
markov-chains markov-process
asked Jan 21 at 21:56
user120911user120911
231110
$endgroup$
$begingroup$
Can you be more specific about what you're trying to do? Matlab does have facilities for a sparse matrix. Of course you should avoid computing anything that might give you a non-sparse matrix, such as $(I - A)^{-1}$.
$endgroup$
– Robert Israel
Jan 21 at 22:04
1
$begingroup$
This question is flagged as off-topic, so you might want to consider editing and explaining what exactly you are asking in a more clear context.
$endgroup$
– onurcanbektas
Jan 22 at 6:19
add a comment |
$begingroup$
Beyond some size of the transition matrix, my computer cannot cope with the Markov Chain problem I am working on (in MATLAB). However, I am sure I am not aware of all the useful tricks that can extend the size of what is possible.
What are some must-know algorithms for working with a very large transition matrix?
A little more specifically, I am working on a problem about the transitions of a group from state to state, where the current state is defined by the configuration of elements in the group (think 1's and 0's for each element, although the state of each element may be more refined) and the transition depends on how the elements in the group change (between 1 and 0 in the simplest case)
Since the combinatorics very quickly blows-up when the group contain many elements and when we move beyond the simple 1 and 0 possible element states, I am having a difficult time examining the behaviour of the group for anything but small groups and simple possible states of elements.
I would like to examine large system behaviour, and therefore I am asking for strategies to push the limits.
It may also be important to note that some states of the group are absorbing states (in which elements in the group stop changing).
markov-chains markov-process
asked Jan 21 at 21:56
user120911user120911
231110
$endgroup$
Beyond some size of the transition matrix, my computer cannot cope with the Markov Chain problem I am working on (in MATLAB). However, I am sure I am not aware of all the useful tricks that can extend the size of what is possible.
What are some must-know algorithms for working with a very large transition matrix?
A little more specifically, I am working on a problem about the transitions of a group from state to state, where the current state is defined by the configuration of elements in the group (think 1's and 0's for each element, although the state of each element may be more refined) and the transition depends on how the elements in the group change (between 1 and 0 in the simplest case)
Since the combinatorics very quickly blows-up when the group contain many elements and when we move beyond the simple 1 and 0 possible element states, I am having a difficult time examining the behaviour of the group for anything but small groups and simple possible states of elements.
I would like to examine large system behaviour, and therefore I am asking for strategies to push the limits.
It may also be important to note that some states of the group are absorbing states (in which elements in the group stop changing).
markov-chains markov-process
markov-chains markov-process
asked Jan 21 at 21:56
user120911user120911
231110
asked Jan 21 at 21:56
user120911user120911
231110
edited Jan 21 at 22:26
user120911
asked Jan 21 at 21:56
user120911user120911
231110
asked Jan 21 at 21:56
user120911user120911
231110
asked Jan 21 at 21:56
user120911user120911
231110
231110
$begingroup$
Can you be more specific about what you're trying to do? Matlab does have facilities for a sparse matrix. Of course you should avoid computing anything that might give you a non-sparse matrix, such as $(I - A)^{-1}$.
$endgroup$
– Robert Israel
Jan 21 at 22:04
1
$begingroup$
This question is flagged as off-topic, so you might want to consider editing and explaining what exactly you are asking in a more clear context.
$endgroup$
– onurcanbektas
Jan 22 at 6:19
add a comment |
$begingroup$
Can you be more specific about what you're trying to do? Matlab does have facilities for a sparse matrix. Of course you should avoid computing anything that might give you a non-sparse matrix, such as $(I - A)^{-1}$.
$endgroup$
– Robert Israel
Jan 21 at 22:04
1
$begingroup$
This question is flagged as off-topic, so you might want to consider editing and explaining what exactly you are asking in a more clear context.
$endgroup$
– onurcanbektas
Jan 22 at 6:19
$begingroup$
Can you be more specific about what you're trying to do? Matlab does have facilities for a sparse matrix. Of course you should avoid computing anything that might give you a non-sparse matrix, such as $(I - A)^{-1}$.
$endgroup$
– Robert Israel
Jan 21 at 22:04
$begingroup$
Can you be more specific about what you're trying to do? Matlab does have facilities for a sparse matrix. Of course you should avoid computing anything that might give you a non-sparse matrix, such as $(I - A)^{-1}$.
$endgroup$
– Robert Israel
Jan 21 at 22:04
1
1
$begingroup$
This question is flagged as off-topic, so you might want to consider editing and explaining what exactly you are asking in a more clear context.
$endgroup$
– onurcanbektas
Jan 22 at 6:19
$begingroup$
This question is flagged as off-topic, so you might want to consider editing and explaining what exactly you are asking in a more clear context.
$endgroup$
– onurcanbektas
Jan 22 at 6:19
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3082475%2flarge-sparse-transition-matrix-strategies-for-computing-given-memory-limitati%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3082475%2flarge-sparse-transition-matrix-strategies-for-computing-given-memory-limitati%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
Can you be more specific about what you're trying to do? Matlab does have facilities for a sparse matrix. Of course you should avoid computing anything that might give you a non-sparse matrix, such as $(I - A)^{-1}$.
$endgroup$
– Robert Israel
Jan 21 at 22:04
1
$begingroup$
This question is flagged as off-topic, so you might want to consider editing and explaining what exactly you are asking in a more clear context.
$endgroup$
– onurcanbektas
Jan 22 at 6:19