Ignore '️' while tokenizing tweets
I'm making a twitter web crawler for sentiment analysis.
I'm following this tutorial https://marcobonzanini.com/2015/03/23/mining-twitter-data-with-python-part-4-rugby-and-term-co-occurrences/.
In this tutorial(part 3) Marco teaches how to ignore some terms using a stop variable in his algorithm. However when I start to collect the tweets and try to match the co-occurrence matrix - which contains the number of times the term x has been seen in the same tweet as the term y - tweepy is collecting this term: '️' . I don't know what this is in Unicode and how can I ignore.
I've tried the apostrophe, single right/left quote, and none of these seems to work.
Any thoughts?
python twitter tweepy sentiment-analysis
asked Nov 13 '18 at 2:18
S. WallaceS. Wallace
105
add a comment |
I'm making a twitter web crawler for sentiment analysis.
I'm following this tutorial https://marcobonzanini.com/2015/03/23/mining-twitter-data-with-python-part-4-rugby-and-term-co-occurrences/.
In this tutorial(part 3) Marco teaches how to ignore some terms using a stop variable in his algorithm. However when I start to collect the tweets and try to match the co-occurrence matrix - which contains the number of times the term x has been seen in the same tweet as the term y - tweepy is collecting this term: '️' . I don't know what this is in Unicode and how can I ignore.
I've tried the apostrophe, single right/left quote, and none of these seems to work.
Any thoughts?
python twitter tweepy sentiment-analysis
asked Nov 13 '18 at 2:18
S. WallaceS. Wallace
105
Array example:[('Centre', '️', 2), ('ES', '️', 2), ('Ridge', '️', 2), ('Cyber', '️', 2), ('Club', '️', 2)]
– S. Wallace
Nov 13 '18 at 2:36
I found the solution. After putting the char in ord() function I found his unicode code, which was 65039 or u"uFE0F". I put in the ignore array and worked!!!
– S. Wallace
Nov 13 '18 at 17:48
add a comment |
I'm making a twitter web crawler for sentiment analysis.
I'm following this tutorial https://marcobonzanini.com/2015/03/23/mining-twitter-data-with-python-part-4-rugby-and-term-co-occurrences/.
In this tutorial(part 3) Marco teaches how to ignore some terms using a stop variable in his algorithm. However when I start to collect the tweets and try to match the co-occurrence matrix - which contains the number of times the term x has been seen in the same tweet as the term y - tweepy is collecting this term: '️' . I don't know what this is in Unicode and how can I ignore.
I've tried the apostrophe, single right/left quote, and none of these seems to work.
Any thoughts?
python twitter tweepy sentiment-analysis
asked Nov 13 '18 at 2:18
S. WallaceS. Wallace
105
I'm making a twitter web crawler for sentiment analysis.
I'm following this tutorial https://marcobonzanini.com/2015/03/23/mining-twitter-data-with-python-part-4-rugby-and-term-co-occurrences/.
In this tutorial(part 3) Marco teaches how to ignore some terms using a stop variable in his algorithm. However when I start to collect the tweets and try to match the co-occurrence matrix - which contains the number of times the term x has been seen in the same tweet as the term y - tweepy is collecting this term: '️' . I don't know what this is in Unicode and how can I ignore.
I've tried the apostrophe, single right/left quote, and none of these seems to work.
Any thoughts?
python twitter tweepy sentiment-analysis
python twitter tweepy sentiment-analysis
asked Nov 13 '18 at 2:18
S. WallaceS. Wallace
105
asked Nov 13 '18 at 2:18
S. WallaceS. Wallace
105
edited Nov 13 '18 at 2:34
S. Wallace
asked Nov 13 '18 at 2:18
S. WallaceS. Wallace
105
asked Nov 13 '18 at 2:18
S. WallaceS. Wallace
105
asked Nov 13 '18 at 2:18
S. WallaceS. Wallace
105
105
Array example:[('Centre', '️', 2), ('ES', '️', 2), ('Ridge', '️', 2), ('Cyber', '️', 2), ('Club', '️', 2)]
– S. Wallace
Nov 13 '18 at 2:36
I found the solution. After putting the char in ord() function I found his unicode code, which was 65039 or u"uFE0F". I put in the ignore array and worked!!!
– S. Wallace
Nov 13 '18 at 17:48
add a comment |
Array example:[('Centre', '️', 2), ('ES', '️', 2), ('Ridge', '️', 2), ('Cyber', '️', 2), ('Club', '️', 2)]
– S. Wallace
Nov 13 '18 at 2:36
I found the solution. After putting the char in ord() function I found his unicode code, which was 65039 or u"uFE0F". I put in the ignore array and worked!!!
– S. Wallace
Nov 13 '18 at 17:48
Array example:
[('Centre', '️', 2), ('ES', '️', 2), ('Ridge', '️', 2), ('Cyber', '️', 2), ('Club', '️', 2)]– S. Wallace
Nov 13 '18 at 2:36
Array example:
[('Centre', '️', 2), ('ES', '️', 2), ('Ridge', '️', 2), ('Cyber', '️', 2), ('Club', '️', 2)]– S. Wallace
Nov 13 '18 at 2:36
I found the solution. After putting the char in ord() function I found his unicode code, which was 65039 or u"uFE0F". I put in the ignore array and worked!!!
– S. Wallace
Nov 13 '18 at 17:48
I found the solution. After putting the char in ord() function I found his unicode code, which was 65039 or u"uFE0F". I put in the ignore array and worked!!!
– S. Wallace
Nov 13 '18 at 17:48
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53272823%2fignore-%25ef%25b8%258f-while-tokenizing-tweets%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53272823%2fignore-%25ef%25b8%258f-while-tokenizing-tweets%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Array example:
[('Centre', '️', 2), ('ES', '️', 2), ('Ridge', '️', 2), ('Cyber', '️', 2), ('Club', '️', 2)]– S. Wallace
Nov 13 '18 at 2:36
I found the solution. After putting the char in ord() function I found his unicode code, which was 65039 or u"uFE0F". I put in the ignore array and worked!!!
– S. Wallace
Nov 13 '18 at 17:48