Odtnhj

I am trying to collect data from twitter using tweepy. I have been able to get the usernames timeline and store it in csv file but, I would like to get new update and also update my csv file when there is an update in the username's timeline.

(I am using python3)

The code I have for collecting data from username time line is:

def get_tweets(api, username, limit):
alltweet =
fid2 = open('_3200unfilteredTweets.csv','w')

""" Download Tweets from username account """
for status in tqdm(tweepy.Cursor(api.user_timeline, screen_name=username,tweet_mode='extended').items(limit),
 unit="tw", total=limit):

 alltweet.append(status.full_text)
# store(status._json) 
 process_tweet(status)
for line in alltweet:
 fid2.write("%sn"%line)

Also the following code allow me to get tweets between certain time , however since I have several users to collect data I can not recall the last time I get tweets.

startDate = datetime(2011, 6, 1, 0, 0, 0)
endDate = datetime(2012, 1, 1, 0, 0, 0)

tweets = 
tmpTweets = api.user_timeline(username)
for tweet in tmpTweets:
 if tweet.created_at < endDate and tweet.created_at > startDate:
 tweets.append(tweet)

while (tmpTweets[-1].created_at > startDate):
 tmpTweets = api.user_timeline(username, max_id = tmpTweets[-1].id)
 for tweet in tmpTweets:
 if tweet.created_at < endDate and tweet.created_at > startDate:
 tweets.append(tweet)

Please let me know if there is any way that I can only get the timeline updates.

Yes, there is a way to do this.

When you first get your list of Tweets, take a note of the ID of the most recent Tweet.

The next time you call user_timeline you can add since_id=....

That will get all the Tweets which have been posted after the ID of your most recent Tweet.

There are some limits though:

Returns results with an ID greater than (that is, more recent than) the specified ID. There are limits to the number of Tweets that can be accessed through the API. If the limit of Tweets has occured since the since_id, the since_id will be forced to the oldest ID available.

You should check out the streaming options that the Twitter API and tweepy offers.
https://tweepy.readthedocs.io/en/v3.5.0/streaming_how_to.html