Ultimately testing and looking at the code is your best way to know the correct answer, but here is some advice:
Without the use of an existing index, when you filter, it will need to iterate through the entire dataset to apply the filter. If you think about it that makes sense. The only time filtering (or indexing) provide value is if you are going to use the results more than once. With an in-memory dataset that is usually not the case.
It will really depend on your use case. If you will only ever do a single iteration pass then filtering or indexing don't make sense. You might consider where you are pulling the data from and what that process looks like. Maybe you can update it during that process.