[WIP] Determinization #37
Conversation
|
OK, thanks. I'll implement |
|
OK, this code is basically finished, but I haven't tried to compile it. |
|
OK, will do |
| state_id(state_id), | ||
| arc_id(incoming_arc_index), | ||
| prev_state(src), | ||
| forward_prob(element->forward_prob + arc_weight) { } |
There was a problem hiding this comment.
src->forward_prob, right?
| argue that this representation is unique. (If not, it won't matter actually; | ||
| it will just give us an output that's less minimal than it could be). | ||
| shared_ptr<LogSumTracebackState> prev_state; | ||
| // `prev_state` is the state that this points back to. |
There was a problem hiding this comment.
prev_state(src) instead of src(src), right?
https://github.com/danpovey/k2/blob/a79f527e32d6fe064f704e0987c263bbf7894e16/k2/csrc/determinize.cc#L237-L240
| // pair<int32_t, float> for LogSumTracebackState. | ||
|
|
||
| // Constructor for the initial state of the determinized FSA | ||
| DetState(): seq_len(0), output_state(-1), normalized(true) { |
There was a problem hiding this comment.
what is output_state, it seems it's never used in DetState?
| state->Normalize(wfsa_in, &arc_weight, &deriv_info); | ||
| int32_t next_state_id; | ||
| bool is_new_state = state_map->GetOutputState(state); | ||
| arcs_out->push_back({this->state_id, next_state_id, label}); |
There was a problem hiding this comment.
what does this point to? what's next_state_id, seems we can remove this function as there is a function Normalize in DetState?
There was a problem hiding this comment.
mm.. maybe this function should not exist but should be inline code, if it's called only once now. this->state_id was the state id of the parent state, which is no longer in scope (was in ProcessArcs()). next_state_id should be state->state_id.
There was a problem hiding this comment.
actually, possibly this function is never called and can be deleted.
| det_state->Normalize(wfsa_in, &arc_weight, &deriv_info); | ||
| if (det_state->forward_backward_prob >= prune_cutoff) { | ||
| bool is_new_state = state_map->GetOutputState(state); | ||
| arcs_out->push_back({this->state_id, next_state_id, label}); |
There was a problem hiding this comment.
GetOutputState(det_state), right?
what's the new_state_id and label
There was a problem hiding this comment.
yes, GetOutputState(det_state). next_state_id == state->state_id. You can get label as iter->first.
| arc_derivs_out->clear(); | ||
|
|
||
| bool ans = map.GetOutputState(start_state.get()); | ||
| CHECK(ans && ans->state_id == 0); |
There was a problem hiding this comment.
can I remove ans->state_id?
|
Can we close this one since it has already been merged in #41 ? |
This is still a work in progress, but closer to being finished.
Putting up a PR so you guys can see the code, it might give you some implementation ideas for things like epsilon removal.