@yourfriendkarol said in solution for massive database data check other than select db by criteria (high I/O)?:
What I do is:
get some data
select by criteria db action, it gives u list
if length of that list > 0 it means record is in db already - continue. If its not there, then it adds.
its good way to do it but i think with many threads my I/O is on max and it gets kinda slow, but maybe im wrong and its something else slowing me.
i basically have such thing as i described above running 24/7 and cant speed it up for some reason nomatter how many threads I add.
CPU, memory, disk, is all ok on low usage, only I/O high
This all depends, is the data within the table long/a lot? 1million + with a high character rate?
Have you tried other Database actions and/or filters to minimize the search criteria?
Would an external MongoDB you set-up & configure yourself might change the I/O bottleneck?
External DB: http://wiki.bablosoft.com/doku.php?id=how_to_setup_mongodb_server
@andykaufseo I didn't read all thread, but here is answer for original question.
You can insert only to new group, which is created with Create Group action.
Create Group returns group id, which can be used later inside Insert Record action.
There will be interface to search/edit/delete groups but later.
Hope that really answered your problem. Can also use the "Select Records" command, change the Table to the second one and add any filters you need. This will return all records in a list and using the Foreach will get the job done.