Odtnhj

I am trying to remove lines in a file that start with the same 5 characters, however, the first 5 characters are random (I don't know what they will be)?

I have a code that reads the last 5 characters of the first line of a file and matches them to the FIRST 5 characters on a random line in the file that has the same 5 characters. The problem is, when there are two or more matches that have the same first 5 characters the code messes up. I need something that reads all the lines in the file and removes one of the two lines that have the same 5 first characters.

Example (issue):

CCTGGATGGCTTATATAAGAT***GTTAT***

***GTTAT***ATAATATACCACCGGGCTGCTT

***GTTAT***ATAGTTACAGCGGAGTCTTGTGACTGGCTCGAGTCAAAAT

What I need as result after one is taken out of file:

CCTGGATGGCTTATATAAGAT***GTTAT***

***GTTAT***ATAATATACCACCGGGCTGCTT

(no third line)

I will greatly appreciate it if you could explain how I could go about this with words as well.

You can do this for example like so:

FILE_NAME = "data.txt" # the name of the file to read in
NR_MATCHING_CHARS = 5 # the number of characters that need to match

lines = set() # a set of lines that contain the beginning of the lines that have already been outputted
with open(FILE_NAME, "r") as inF: # open the file
 for line in inF: # for every line
 line = line.strip() # that is
 if line == "": continue # not empty
 beginOfSequence = line[:NR_MATCHING_CHARS]
 if not (beginOfSequence in lines): # and the beginning of this line was not printed yet
 print(line) # print the line
 lines.add(beginOfSequence) # remember that the beginning of the line