Apache Flink: Is MapState automatically updated when I modify a stored object?
Is it necessary to use MapState.put() to manually update the state or whether is the state automatically updated when I modify an object?
private transient MapState<String, Word> words;
.......
Word w = words.get(word);
if (w == null)
w = new Word(word);
//words.put(word, w); //A
if (....)
w.countBad(1); // countXXX modifies a the private variable in a Word object
else
w.countGood(1);
//words.put(word, w); //B
Q: If I use the A method, will the next count calculation automatically update the corresponding Mapstate state? Or do I need to use the B method to manually update the state after the calculation is complete?
apache-flink flink-streaming
edited Nov 12 at 11:02
Fabian Hueske
11.9k12027
asked Nov 12 at 7:00
Cheng Jiang
296
add a comment |
Is it necessary to use MapState.put() to manually update the state or whether is the state automatically updated when I modify an object?
private transient MapState<String, Word> words;
.......
Word w = words.get(word);
if (w == null)
w = new Word(word);
//words.put(word, w); //A
if (....)
w.countBad(1); // countXXX modifies a the private variable in a Word object
else
w.countGood(1);
//words.put(word, w); //B
Q: If I use the A method, will the next count calculation automatically update the corresponding Mapstate state? Or do I need to use the B method to manually update the state after the calculation is complete?
apache-flink flink-streaming
edited Nov 12 at 11:02
Fabian Hueske
11.9k12027
asked Nov 12 at 7:00
Cheng Jiang
296
add a comment |
Is it necessary to use MapState.put() to manually update the state or whether is the state automatically updated when I modify an object?
private transient MapState<String, Word> words;
.......
Word w = words.get(word);
if (w == null)
w = new Word(word);
//words.put(word, w); //A
if (....)
w.countBad(1); // countXXX modifies a the private variable in a Word object
else
w.countGood(1);
//words.put(word, w); //B
Q: If I use the A method, will the next count calculation automatically update the corresponding Mapstate state? Or do I need to use the B method to manually update the state after the calculation is complete?
apache-flink flink-streaming
edited Nov 12 at 11:02
Fabian Hueske
11.9k12027
asked Nov 12 at 7:00
Cheng Jiang
296
Is it necessary to use MapState.put() to manually update the state or whether is the state automatically updated when I modify an object?
private transient MapState<String, Word> words;
.......
Word w = words.get(word);
if (w == null)
w = new Word(word);
//words.put(word, w); //A
if (....)
w.countBad(1); // countXXX modifies a the private variable in a Word object
else
w.countGood(1);
//words.put(word, w); //B
Q: If I use the A method, will the next count calculation automatically update the corresponding Mapstate state? Or do I need to use the B method to manually update the state after the calculation is complete?
apache-flink flink-streaming
apache-flink flink-streaming
edited Nov 12 at 11:02
Fabian Hueske
11.9k12027
asked Nov 12 at 7:00
Cheng Jiang
296
edited Nov 12 at 11:02
Fabian Hueske
11.9k12027
asked Nov 12 at 7:00
Cheng Jiang
296
edited Nov 12 at 11:02
Fabian Hueske
11.9k12027
edited Nov 12 at 11:02
Fabian Hueske
11.9k12027
edited Nov 12 at 11:02
Fabian Hueske
11.9k12027
11.9k12027
asked Nov 12 at 7:00
Cheng Jiang
296
asked Nov 12 at 7:00
Cheng Jiang
296
asked Nov 12 at 7:00
Cheng Jiang
296
296
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
From an API point of view, you always need to manually update the state.
However, the actual behavior depends on the state backend. If the application uses the InMemoryStateBackend or the FsStateBackend, all local state is stored on the JVM heap of the worker process, i.e., the state backend just holds a reference to the object. Hence, the state is directly modified when you modify the object.
If you use the RocksDBStateBackend all state accesses are de/serialized and read from / written to RocksDB. In this case modifying the object does not have an effect on the state.
I recommend to always explicitly update the state because this will ensure that you can switch the state backend without adjusting the logic of your application.
answered Nov 12 at 10:57
Fabian Hueske
11.9k12027
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53257240%2fapache-flink-is-mapstate-automatically-updated-when-i-modify-a-stored-object%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
From an API point of view, you always need to manually update the state.
However, the actual behavior depends on the state backend. If the application uses the InMemoryStateBackend or the FsStateBackend, all local state is stored on the JVM heap of the worker process, i.e., the state backend just holds a reference to the object. Hence, the state is directly modified when you modify the object.
If you use the RocksDBStateBackend all state accesses are de/serialized and read from / written to RocksDB. In this case modifying the object does not have an effect on the state.
I recommend to always explicitly update the state because this will ensure that you can switch the state backend without adjusting the logic of your application.
answered Nov 12 at 10:57
Fabian Hueske
11.9k12027
add a comment |
From an API point of view, you always need to manually update the state.
However, the actual behavior depends on the state backend. If the application uses the InMemoryStateBackend or the FsStateBackend, all local state is stored on the JVM heap of the worker process, i.e., the state backend just holds a reference to the object. Hence, the state is directly modified when you modify the object.
If you use the RocksDBStateBackend all state accesses are de/serialized and read from / written to RocksDB. In this case modifying the object does not have an effect on the state.
I recommend to always explicitly update the state because this will ensure that you can switch the state backend without adjusting the logic of your application.
answered Nov 12 at 10:57
Fabian Hueske
11.9k12027
add a comment |
From an API point of view, you always need to manually update the state.
However, the actual behavior depends on the state backend. If the application uses the InMemoryStateBackend or the FsStateBackend, all local state is stored on the JVM heap of the worker process, i.e., the state backend just holds a reference to the object. Hence, the state is directly modified when you modify the object.
If you use the RocksDBStateBackend all state accesses are de/serialized and read from / written to RocksDB. In this case modifying the object does not have an effect on the state.
I recommend to always explicitly update the state because this will ensure that you can switch the state backend without adjusting the logic of your application.
answered Nov 12 at 10:57
Fabian Hueske
11.9k12027
From an API point of view, you always need to manually update the state.
However, the actual behavior depends on the state backend. If the application uses the InMemoryStateBackend or the FsStateBackend, all local state is stored on the JVM heap of the worker process, i.e., the state backend just holds a reference to the object. Hence, the state is directly modified when you modify the object.
If you use the RocksDBStateBackend all state accesses are de/serialized and read from / written to RocksDB. In this case modifying the object does not have an effect on the state.
I recommend to always explicitly update the state because this will ensure that you can switch the state backend without adjusting the logic of your application.
answered Nov 12 at 10:57
Fabian Hueske
11.9k12027
answered Nov 12 at 10:57
Fabian Hueske
11.9k12027
answered Nov 12 at 10:57
Fabian Hueske
11.9k12027
answered Nov 12 at 10:57
Fabian Hueske
11.9k12027
11.9k12027
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53257240%2fapache-flink-is-mapstate-automatically-updated-when-i-modify-a-stored-object%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
